Sep 10 21:10:22.950612 ip-10-0-135-74 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 21:10:23.331845 ip-10-0-135-74 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:10:23.331845 ip-10-0-135-74 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 21:10:23.331845 ip-10-0-135-74 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:10:23.331845 ip-10-0-135-74 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 21:10:23.331845 ip-10-0-135-74 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:10:23.332676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.332584    2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 21:10:23.337374 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337354    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:23.337374 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337372    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:23.337374 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337376    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:23.337374 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337380    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337384    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337387    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337390    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337394    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337397    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337400    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337402    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337405    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337408    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337411    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337413    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337416    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337419    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337422    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337424    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337427    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337429    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337432    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337435    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:23.337517 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337437    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337441    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337443    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337446    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337449    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337452    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337456    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337458    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337461    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337463    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337466    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337469    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337472    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337475    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337478    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337481    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337483    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337486    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337488    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337491    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:23.338003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337494    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337496    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337499    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337502    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337505    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337508    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337511    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337514    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337516    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337519    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337524    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337528    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337531    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337534    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337536    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337539    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337542    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337545    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337548    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337551    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:23.338498 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337553    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337556    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337559    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337561    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337564    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337566    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337569    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337572    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337574    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337577    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337579    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337582    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337585    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337587    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337591    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337593    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337596    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337600    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337603    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337608    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:23.339000 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337611    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337613    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337618    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337620    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337623    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337626    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337629    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337633    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337636    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337639    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337642    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337645    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337648    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337651    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337655    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.337657    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338390    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338398    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338402    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:23.339500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338405    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338408    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338411    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338415    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338417    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338420    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338423    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338425    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338428    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338431    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338433    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338436    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338439    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338441    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338444    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338447    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338450    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338452    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338455    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338458    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:23.339959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338460    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338463    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338466    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338469    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338472    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338474    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338478    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338480    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338482    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338485    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338487    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338490    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338492    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338497    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338500    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338504    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338507    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338509    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338512    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338515    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:23.340464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338519    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338522    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338525    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338528    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338531    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338533    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338537    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338541    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338543    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338547    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338549    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338552    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338556    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338560    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338563    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338566    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338569    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338571    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338574    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:23.340958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338577    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338580    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338582    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338585    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338588    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338591    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338593    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338596    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338598    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338601    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338603    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338605    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338609    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338611    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338614    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338616    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338619    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338621    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338624    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338627    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:23.341447 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338629    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338631    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338634    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338637    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338639    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338641    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338644    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338647    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338650    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338652    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338655    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338657    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338660    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338663    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338666    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338668    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.338671    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339064    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339070    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339074    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:23.341937 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339077    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339080    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339083    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339086    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339090    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339093    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339096    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339099    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339102    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339105    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339108    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339111    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339113    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339116    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339119    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339122    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339126    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339128    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339131    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:23.342463 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339134    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339137    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339139    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339142    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339145    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339148    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339151    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339154    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339157    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339160    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339162    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339165    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339167    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339170    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339172    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339174    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339177    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339180    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339183    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339186    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:23.342928 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339188    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339191    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339193    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339196    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339198    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339201    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339203    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339206    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339208    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339211    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339213    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339216    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339219    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339221    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339224    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339226    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339229    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339232    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339234    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339237    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:23.343436 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339240    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339243    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339245    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339248    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339251    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339254    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339256    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339259    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339262    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339264    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339282    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339286    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339289    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339292    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339294    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339297    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339299    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339302    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339305    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339307    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:23.343965 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339310    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339312    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339315    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339318    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339320    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339323    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339326    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339329    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339331    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339334    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339338    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339340    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339344    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339347    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339351    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339354    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.339357    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339443    2573 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339479    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339486    2573 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 21:10:23.344464 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339492    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339497    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339500    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339505    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339510    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339513    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339517    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339520    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339524    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339527    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339530    2573 flags.go:64] FLAG: --cgroup-root=""
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339533    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339537    2573 flags.go:64] FLAG: --client-ca-file=""
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339539    2573 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339545    2573 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339549    2573 flags.go:64] FLAG: --cluster-domain=""
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339552    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339556    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339559    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339562    2573 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339566    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339570    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339573    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 21:10:23.344946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339577    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339580    2573 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339584    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339587    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339591    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339594    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339598    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339601    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339604    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339607    2573 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339610    2573 flags.go:64] FLAG: --enable-server="true"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339613    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339616    2573 flags.go:64] FLAG: --event-burst="100"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339620    2573 flags.go:64] FLAG: --event-qps="50"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339622    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339626    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339629    2573 flags.go:64] FLAG: --eviction-hard=""
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339633    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339636    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339640    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339643    2573 flags.go:64] FLAG: --eviction-soft=""
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339646    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339649    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339653    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339657    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 21:10:23.345562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339660    2573 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339662    2573 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339665    2573 flags.go:64] FLAG: --feature-gates=""
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339669    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339673    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339678    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339681    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339685    2573 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339688    2573 flags.go:64] FLAG: --help="false"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339691    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339695    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339698    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339701    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339705    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339709    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339711    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339714    2573 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339717    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339720    2573 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339723    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339726    2573 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339729    2573 flags.go:64] FLAG: --kube-reserved=""
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339732    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339735    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 21:10:23.346313 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339738    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339741    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339744    2573 flags.go:64] FLAG: --lock-file=""
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339747    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339750    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339754    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339761    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339764    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339767    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339770    2573 flags.go:64] FLAG: --logging-format="text"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339772    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339776    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339779    2573 flags.go:64] FLAG: --manifest-url=""
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339783    2573 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339788    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339791    2573 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339795    2573 flags.go:64] FLAG: --max-pods="110"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339798    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339801    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339805    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339808    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339811    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339814    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339817    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339824    2573 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 21:10:23.346969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339827    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339830    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339834    2573 flags.go:64] FLAG: --pod-cidr=""
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339837    2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339842    2573 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339845    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339848    2573 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339851    2573 flags.go:64] FLAG: --port="10250"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339854    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339857    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f51c0ca4da7af339"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339860    2573 flags.go:64] FLAG: --qos-reserved=""
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339863    2573 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339866    2573 flags.go:64] FLAG: --register-node="true"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339869    2573 flags.go:64] FLAG: --register-with-taints=""
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339873    2573 flags.go:64] FLAG: --registry-burst="10"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339877    2573 flags.go:64] FLAG: --registry-qps="5"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339880    2573 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339883    2573 flags.go:64] FLAG: --reserved-memory=""
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339886    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339890    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339894    2573 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339897    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339900    2573 flags.go:64] FLAG: --runonce="false"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339903    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339906    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 21:10:23.347685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339909    2573 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339912    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339916    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339919    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339922    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339925    2573 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339928    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339931    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339934    2573 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339937    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339940    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339943    2573 flags.go:64] FLAG: --system-cgroups=""
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339946    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339951    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339954    2573 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339957    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339962    2573 flags.go:64] FLAG: --tls-min-version=""
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339965    2573 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339968    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339971    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339974    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339978    2573 flags.go:64] FLAG: --v="2"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339983    2573 flags.go:64] FLAG: --version="false"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339987    2573 flags.go:64] FLAG: --vmodule=""
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339991    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 21:10:23.348421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.339994    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340113    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340119    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340122    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340125    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340128    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340131    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340134    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340137    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340140    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340153    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340155    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340158    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340161    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340164    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340166    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340169    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340171    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340174    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340177    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:23.349368 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340180    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340182    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340185    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340187    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340190    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340193    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340196    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340200    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340205    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340209    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340212    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340215    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340218    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340221    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340226    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340229    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340232    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340235    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340237    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:23.349958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340240    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340242    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340245    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340248    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340251    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340253    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340256    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340259    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340261    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340264    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340279    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340286    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340291    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340294    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340297    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340300    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340303    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340306    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340309    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:23.350453 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340312    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340314    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340319    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340322    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340324    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340327    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340330    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340333    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340338    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340341    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340343    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340346    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340349    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340351    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340354    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340357    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340360    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340363    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340368    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:23.350947 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340371    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340374    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340376    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340379    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340382    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340385    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340387    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340390    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340392    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340395    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340397    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340400    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340403    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340405    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340408    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340412    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340414    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340417    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340420    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340423    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:23.351425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340426    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340429    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.340432    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.340926    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.349878    2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.349896    2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.349911    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.349917    2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.349979    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.349985    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.349991    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.349996    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.349999    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350002    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:23.351920 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350005    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350009    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350012    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350015    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350018    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350021    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350024    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350027    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350030    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350034    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350036    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350039    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350042    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350044    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350047    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350049    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350052    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350055    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350057    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350060    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:23.352263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350063    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350066    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350068    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350072    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350075    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350078    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350080    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350083    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350085    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350088    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350091    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350093    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350096    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350098    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350101    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350103    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350106    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350109    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350111    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:23.352793 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350114    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350119    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350123    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350126    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350129    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350132    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350135    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350138    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350140    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350143    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350147    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350149    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350152    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350155    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350158    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350160    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350164    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350167    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350169    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:23.353430 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350172    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350175    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350178    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350180    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350183    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350186    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350188    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350191    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350194    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350197    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350199    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350202    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350205    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350207    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350210    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350213    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350215    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350218    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350221    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350223    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:23.353912 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350226    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350228    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350231    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350233    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350236    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350238    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350241    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350244    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350246    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350250    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350253    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350256    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350259    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350262    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350265    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:23.354411 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.350288    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350406    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350411    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350415    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350418    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350421    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350424    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350427    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350430    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350432    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350435    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350439    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350443    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350447    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350449    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350452    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350455    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350458    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350461    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:23.354797 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350463    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350466    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350469    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350472    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350475    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350477    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350480    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350483    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350486    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350488    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350491    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350494    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350497    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350500    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350502    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350505    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350508    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350510    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350513    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350516    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:23.355256 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350519    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350521    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350524    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350526    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350529    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350532    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350534    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350537    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350540    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350542    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350545    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350548    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350551    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350554    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350558    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350561    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350564    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350567    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350570    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350573    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:23.355766 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350576    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350579    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350582    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350584    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350587    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350590    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350593    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350595    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350598    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350601    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350605    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350607    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350610    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350612    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350615    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350618    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350620    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350623    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350626    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:23.356263 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350628    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350631    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350634    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350636    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350640    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350642    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350645    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350648    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350650    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350653    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350656    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350658    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350661    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350663    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350666    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350668    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350671    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350674    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350676    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350679    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:23.356737 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350682    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:23.357238 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:23.350685    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:23.357238 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.350690    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:10:23.357238 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.350836    2573 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 21:10:23.357238 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.353259    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 21:10:23.357238 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.354218    2573 server.go:1019] "Starting client certificate rotation"
Sep 10 21:10:23.357238 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.354324    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 21:10:23.357238 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.355117    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 21:10:23.357238 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.356925    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 21:10:23.358160 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.358137    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 21:10:23.368942 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.368924    2573 log.go:25] "Validated CRI v1 runtime API"
Sep 10 21:10:23.372215 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.372195    2573 log.go:25] "Validated CRI v1 image API"
Sep 10 21:10:23.373087 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.373073    2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 21:10:23.376771 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.376749    2573 fs.go:135] Filesystem UUIDs: map[58a4dd8f-3ecb-4615-b3b3-d35cad3919dd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 99f43cbc-b299-44e3-8a33-c9ed3fcce000:/dev/nvme0n1p4]
Sep 10 21:10:23.376867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.376770    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 21:10:23.382286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.382154    2573 manager.go:216] Machine: {Timestamp:2026-09-10 21:10:23.38084669 +0000 UTC m=+0.317019996 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100277 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cb6002cdb3baf0b1ba903887a7889 SystemUUID:ec2cb600-2cdb-3baf-0b1b-a903887a7889 BootID:a0bf03eb-9d0d-4f51-86bb-47a13fe69f44 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ba:55:6b:a4:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ba:55:6b:a4:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:19:0e:f8:51:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 21:10:23.382286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.382259    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 21:10:23.382457 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.382396    2573 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 21:10:23.383112 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.383091    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 21:10:23.385327 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.385300    2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 21:10:23.385538 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.385334    2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-74.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 21:10:23.386248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.386237    2573 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 21:10:23.386325 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.386255    2573 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 21:10:23.386325 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.386282    2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 21:10:23.387002 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.386992    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 21:10:23.388748 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.388736    2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 21:10:23.392239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.392224    2573 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 21:10:23.392390 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.392378    2573 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 21:10:23.392460 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.392416    2573 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 21:10:23.392460 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.392442    2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 21:10:23.392546 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.392461    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 21:10:23.392546 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.392475    2573 kubelet.go:398] "Adding apiserver pod source"
Sep 10 21:10:23.392631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.392559    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 21:10:23.393034 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.393021    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 21:10:23.393100 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.393043    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 21:10:23.397198 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.397146    2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 21:10:23.400099 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.400084    2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 21:10:23.400177 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.400114    2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 21:10:23.401860 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401847    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 21:10:23.401920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401864    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 21:10:23.401920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401870    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 21:10:23.401920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401876    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 21:10:23.401920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401885    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 21:10:23.401920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401891    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 21:10:23.401920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401905    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 21:10:23.401920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401910    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 21:10:23.401920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401917    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 21:10:23.401920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401923    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 21:10:23.402199 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401932    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 21:10:23.402199 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401937    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 21:10:23.402199 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401959    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 21:10:23.402199 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.401965    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 21:10:23.403645 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.403628    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-74.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 21:10:23.403929 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.403910    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-74.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 21:10:23.403977 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.403911    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 21:10:23.405928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.405915    2573 server.go:1268] "Started kubelet"
Sep 10 21:10:23.406002 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.405977    2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 21:10:23.406112 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.406068    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 21:10:23.406173 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.406141    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 21:10:23.407106 ip-10-0-135-74 systemd[1]: Started Kubernetes Kubelet.
Sep 10 21:10:23.408518 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.407510    2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 21:10:23.408518 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.407946    2573 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 21:10:23.412324 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.412302    2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 21:10:23.413996 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.413980    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nrpqq"
Sep 10 21:10:23.415400 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.414466    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-74.ec2.internal.18d411e41cc5fc6a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-74.ec2.internal,UID:ip-10-0-135-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-74.ec2.internal,},FirstTimestamp:2026-09-10 21:10:23.405882474 +0000 UTC m=+0.342055782,LastTimestamp:2026-09-10 21:10:23.405882474 +0000 UTC m=+0.342055782,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-74.ec2.internal,}"
Sep 10 21:10:23.419794 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.419779    2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 21:10:23.419852 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.419794    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 21:10:23.420198 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.420182    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:23.420544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420529    2573 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 21:10:23.420625 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420557    2573 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 21:10:23.420625 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420529    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 21:10:23.420794 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420622    2573 factory.go:55] Registering systemd factory
Sep 10 21:10:23.420794 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420647    2573 factory.go:223] Registration of the systemd container factory successfully
Sep 10 21:10:23.420794 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420687    2573 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 21:10:23.420794 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420697    2573 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 21:10:23.420944 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420914    2573 factory.go:153] Registering CRI-O factory
Sep 10 21:10:23.420944 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420927    2573 factory.go:223] Registration of the crio container factory successfully
Sep 10 21:10:23.421042 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420978    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 21:10:23.421042 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.420996    2573 factory.go:103] Registering Raw factory
Sep 10 21:10:23.421042 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.421011    2573 manager.go:1196] Started watching for new ooms in manager
Sep 10 21:10:23.421515 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.421502    2573 manager.go:349] Starting recovery of all containers
Sep 10 21:10:23.424453 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.424429    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nrpqq"
Sep 10 21:10:23.434246 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.434217    2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:23.435364 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.435337    2573 manager.go:354] Recovery completed
Sep 10 21:10:23.439860 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.439847    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:23.441867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.441850    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:23.441958 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.441880    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:23.441958 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.441890    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:23.442155 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.442140    2573 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-74.ec2.internal\" not found" node="ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.442553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.442540    2573 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 21:10:23.442553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.442554    2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 21:10:23.442659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.442573    2573 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 21:10:23.442659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.442576    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:23.443337 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.443318    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:23.443404 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.443373    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:23.443404 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.443388    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:23.589021 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.520772    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:23.589021 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.588410    2573 policy_none.go:49] "None policy: Start"
Sep 10 21:10:23.589021 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.588441    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 21:10:23.589021 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.588453    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 21:10:23.589682 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.589665    2573 policy_none.go:47] "Start"
Sep 10 21:10:23.621369 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.621346    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:23.626944 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.626929    2573 manager.go:337] "Starting Device Plugin manager"
Sep 10 21:10:23.627053 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.626969    2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 21:10:23.627053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.626979    2573 server.go:87] "Starting device plugin registration server"
Sep 10 21:10:23.627374 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.627362    2573 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 21:10:23.627440 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.627375    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 21:10:23.627502 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.627485    2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 21:10:23.627667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.627595    2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 21:10:23.627667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.627608    2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 21:10:23.628182 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.628136    2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 21:10:23.628287 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.628217    2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:23.668587 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.668547    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 21:10:23.669804 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.669791    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 21:10:23.669881 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.669842    2573 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 21:10:23.669881 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.669865    2573 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 21:10:23.669983 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.669968    2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 21:10:23.672533 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.672509    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:23.727928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.727874    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:23.728927 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.728909    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:23.729029 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.728947    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:23.729029 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.728961    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:23.729029 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.728994    2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.738089 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.738064    2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.738191 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.738093    2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-74.ec2.internal\": node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:23.755601 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.755568    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:23.770427 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.770394    2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal"]
Sep 10 21:10:23.770597 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.770485    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:23.772231 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.772214    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:23.772338 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.772249    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:23.772338 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.772265    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:23.773704 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.773686    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:23.773816 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.773801    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.773867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.773832    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:23.775046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.775032    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:23.775046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.775037    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:23.775164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.775058    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:23.775164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.775057    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:23.775164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.775079    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:23.775164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.775069    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:23.776329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.776313    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.776413 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.776343    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:23.777536 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.777515    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:23.777621 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.777542    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:23.777621 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.777553    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:23.793868 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.793842    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-74.ec2.internal\" not found" node="ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.797942 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.797926    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-74.ec2.internal\" not found" node="ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.822226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.822195    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e53f32f7691c310cd328965d8dd7641c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal\" (UID: \"e53f32f7691c310cd328965d8dd7641c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.822226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.822226    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e53f32f7691c310cd328965d8dd7641c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal\" (UID: \"e53f32f7691c310cd328965d8dd7641c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.822478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.822242    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bcd962acf554a8e9018c7c906807b563-config\") pod \"kube-apiserver-proxy-ip-10-0-135-74.ec2.internal\" (UID: \"bcd962acf554a8e9018c7c906807b563\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.856561 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.856477    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:23.922476 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.922435    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e53f32f7691c310cd328965d8dd7641c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal\" (UID: \"e53f32f7691c310cd328965d8dd7641c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.922476 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.922475    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e53f32f7691c310cd328965d8dd7641c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal\" (UID: \"e53f32f7691c310cd328965d8dd7641c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.922713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.922491    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bcd962acf554a8e9018c7c906807b563-config\") pod \"kube-apiserver-proxy-ip-10-0-135-74.ec2.internal\" (UID: \"bcd962acf554a8e9018c7c906807b563\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.922713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.922544    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bcd962acf554a8e9018c7c906807b563-config\") pod \"kube-apiserver-proxy-ip-10-0-135-74.ec2.internal\" (UID: \"bcd962acf554a8e9018c7c906807b563\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.922713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.922543    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e53f32f7691c310cd328965d8dd7641c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal\" (UID: \"e53f32f7691c310cd328965d8dd7641c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.922713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:23.922568    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e53f32f7691c310cd328965d8dd7641c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal\" (UID: \"e53f32f7691c310cd328965d8dd7641c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:23.957580 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:23.957539    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:24.058031 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:24.057996    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:24.097209 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.097180    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:24.102902 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.102878    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:24.158770 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:24.158683    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:24.259211 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:24.259166    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:24.353863 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.353830    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 21:10:24.354456 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.353960    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 21:10:24.354456 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.354014    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 21:10:24.360134 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:24.360114    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-74.ec2.internal\" not found"
Sep 10 21:10:24.395941 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.395911    2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:24.420704 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.420626    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:24.420704 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.420626    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 21:10:24.428336 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.428301    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 21:05:24 +0000 UTC" deadline="2028-06-23 12:54:44.114355148 +0000 UTC"
Sep 10 21:10:24.428336 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.428327    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15639h44m19.686030358s"
Sep 10 21:10:24.429651 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.429638    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 21:10:24.431414 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.431388    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal"
Sep 10 21:10:24.431567 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.431548    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 21:10:24.444062 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.444041    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 21:10:24.456583 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.456550    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4htnb"
Sep 10 21:10:24.463900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.463881    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4htnb"
Sep 10 21:10:24.639128 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:24.639085    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd962acf554a8e9018c7c906807b563.slice/crio-3c95c7c1943c244cd042d34b12c3fd06d6f6569d4ae3f31ed3ebb178f45aa673 WatchSource:0}: Error finding container 3c95c7c1943c244cd042d34b12c3fd06d6f6569d4ae3f31ed3ebb178f45aa673: Status 404 returned error can't find the container with id 3c95c7c1943c244cd042d34b12c3fd06d6f6569d4ae3f31ed3ebb178f45aa673
Sep 10 21:10:24.643522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.643506    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:10:24.670927 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:24.670873    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53f32f7691c310cd328965d8dd7641c.slice/crio-64f96da4a18b80a1d9f8c27df74a209672b13eef95bc248317f8c92e4ac4c078 WatchSource:0}: Error finding container 64f96da4a18b80a1d9f8c27df74a209672b13eef95bc248317f8c92e4ac4c078: Status 404 returned error can't find the container with id 64f96da4a18b80a1d9f8c27df74a209672b13eef95bc248317f8c92e4ac4c078
Sep 10 21:10:24.672988 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:24.672941    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal" event={"ID":"bcd962acf554a8e9018c7c906807b563","Type":"ContainerStarted","Data":"3c95c7c1943c244cd042d34b12c3fd06d6f6569d4ae3f31ed3ebb178f45aa673"}
Sep 10 21:10:25.359860 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.359787    2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:25.393425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.393398    2573 apiserver.go:52] "Watching apiserver"
Sep 10 21:10:25.399711 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.399682    2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 21:10:25.400187 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.400159    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal","openshift-network-operator/iptables-alerter-rlv9n","openshift-ovn-kubernetes/ovnkube-node-bp6vk","kube-system/global-pull-secret-syncer-z8k59","openshift-cluster-node-tuning-operator/tuned-82gdq","openshift-dns/node-resolver-dwlgl","openshift-image-registry/node-ca-svw8v","openshift-multus/multus-additional-cni-plugins-fc4cj","openshift-multus/multus-wh98p","openshift-multus/network-metrics-daemon-vqprq","openshift-network-diagnostics/network-check-target-4zmfv","kube-system/konnectivity-agent-vttxv","kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal"]
Sep 10 21:10:25.401735 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.401706    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.403300 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.403263    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:25.403419 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.403375    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:25.404002 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.403909    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 21:10:25.404112 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.404019    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 21:10:25.404112 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.404026    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gtdsc\""
Sep 10 21:10:25.405067 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.405046    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.406384 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.406364    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.411414 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.408616    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ct6xd\""
Sep 10 21:10:25.411414 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.408891    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 21:10:25.411414 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.408933    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 21:10:25.411414 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.409412    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.411671 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.411624    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 21:10:25.411822 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.411477    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.411933 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.411891    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 21:10:25.411933 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.411901    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 21:10:25.412036 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.411980    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 21:10:25.412036 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.411905    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 21:10:25.412222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.412195    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 21:10:25.413061 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.413039    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fwnjc\""
Sep 10 21:10:25.413153 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.413105    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.413153 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.413147    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 21:10:25.414330 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.414310    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:25.416095 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.416076    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.416986 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.416623    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 21:10:25.416986 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.416632    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 21:10:25.416986 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.416659    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:10:25.416986 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.416721    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 21:10:25.416986 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.416887    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 21:10:25.417751 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.417731    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 21:10:25.417867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.417850    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 21:10:25.417927 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.417861    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 21:10:25.418017 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.417927    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 21:10:25.418073 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.418024    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 21:10:25.418073 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.418034    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 21:10:25.418172 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.418133    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 21:10:25.418219 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.418190    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dwwnr\""
Sep 10 21:10:25.418291 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.418240    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-khssb\""
Sep 10 21:10:25.418444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.418426    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:25.418521 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.418504    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:25.418579 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.418511    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nx2q5\""
Sep 10 21:10:25.418579 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.418545    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 21:10:25.418679 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.418656    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lh2gw\""
Sep 10 21:10:25.418729 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.418688    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-59p66\""
Sep 10 21:10:25.419701 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.419683    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:25.419789 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.419765    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:25.421527 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.421507    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:25.423833 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.423811    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:10:25.423921 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.423838    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 21:10:25.423921 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.423870    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vfdnx\""
Sep 10 21:10:25.424184 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.424105    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 21:10:25.431317 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431288    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa938a91-1541-485a-b88f-151697650d2a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.431433 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431335    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4wzp\" (UniqueName: \"kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp\") pod \"network-check-target-4zmfv\" (UID: \"492738f6-5f08-4101-837f-87cf59c8e113\") " pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:25.431433 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431365    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-socket-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.431433 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431389    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhjbq\" (UniqueName: \"kubernetes.io/projected/3fbc2efe-e578-40d2-8897-ddcb48ef5588-kube-api-access-nhjbq\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.431433 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431417    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-run\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.431658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431438    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e94852a3-19d2-4766-87ac-187802031bf8-etc-tuned\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.431658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431461    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1f372f51-ce6b-46c2-9438-df2f95851903-dbus\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:25.431658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431483    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-var-lib-openvswitch\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.431658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431508    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa938a91-1541-485a-b88f-151697650d2a-cnibin\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.431658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431530    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-sysctl-conf\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.431658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431602    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-socket-dir-parent\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.432041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431670    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-var-lib-kubelet\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.432041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431703    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff0b971f-0f3c-45dd-8234-c68a65df7eae-iptables-alerter-script\") pod \"iptables-alerter-rlv9n\" (UID: \"ff0b971f-0f3c-45dd-8234-c68a65df7eae\") " pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:25.432041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431766    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-kubelet-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.432041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431791    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-etc-selinux\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.432041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431816    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-run-multus-certs\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.432041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431885    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1f372f51-ce6b-46c2-9438-df2f95851903-kubelet-config\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:25.432041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431914    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0e6d31a2-19e9-4970-937a-e30522ffc0df-serviceca\") pod \"node-ca-svw8v\" (UID: \"0e6d31a2-19e9-4970-937a-e30522ffc0df\") " pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.432041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431936    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-kubernetes\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.432041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.431995    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19e148c2-b280-48d0-ac44-7246e35a71ea-cni-binary-copy\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.432041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432030    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-cni-bin\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432103    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f311012a-e71c-45af-b64d-83e2245f52ad-ovnkube-config\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432137    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-sysconfig\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432165    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3160b92d-6922-4d94-a5ed-b92338837c2f-konnectivity-ca\") pod \"konnectivity-agent-vttxv\" (UID: \"3160b92d-6922-4d94-a5ed-b92338837c2f\") " pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432191    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432231    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfc5e2be-0dac-4a19-9c82-f70065c9a9b4-tmp-dir\") pod \"node-resolver-dwlgl\" (UID: \"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4\") " pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432298    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432336    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-sys-fs\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432371    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-modprobe-d\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432406    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-daemon-config\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432468    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-etc-openvswitch\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432494    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff0b971f-0f3c-45dd-8234-c68a65df7eae-host-slash\") pod \"iptables-alerter-rlv9n\" (UID: \"ff0b971f-0f3c-45dd-8234-c68a65df7eae\") " pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432523    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e94852a3-19d2-4766-87ac-187802031bf8-tmp\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.432539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432546    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-hostroot\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432606    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-conf-dir\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432629    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-node-log\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432663    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w49d\" (UniqueName: \"kubernetes.io/projected/f311012a-e71c-45af-b64d-83e2245f52ad-kube-api-access-8w49d\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432709    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mn5p\" (UniqueName: \"kubernetes.io/projected/fa938a91-1541-485a-b88f-151697650d2a-kube-api-access-5mn5p\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432740    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-systemd\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432764    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s76k9\" (UniqueName: \"kubernetes.io/projected/e94852a3-19d2-4766-87ac-187802031bf8-kube-api-access-s76k9\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432821    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3160b92d-6922-4d94-a5ed-b92338837c2f-agent-certs\") pod \"konnectivity-agent-vttxv\" (UID: \"3160b92d-6922-4d94-a5ed-b92338837c2f\") " pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432849    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-run-openvswitch\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432875    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432899    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-cnibin\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432930    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-var-lib-cni-multus\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432952    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbxsr\" (UniqueName: \"kubernetes.io/projected/19e148c2-b280-48d0-ac44-7246e35a71ea-kube-api-access-tbxsr\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.432978    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f311012a-e71c-45af-b64d-83e2245f52ad-ovn-node-metrics-cert\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433011    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e6d31a2-19e9-4970-937a-e30522ffc0df-host\") pod \"node-ca-svw8v\" (UID: \"0e6d31a2-19e9-4970-937a-e30522ffc0df\") " pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433033    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m8qs\" (UniqueName: \"kubernetes.io/projected/0e6d31a2-19e9-4970-937a-e30522ffc0df-kube-api-access-7m8qs\") pod \"node-ca-svw8v\" (UID: \"0e6d31a2-19e9-4970-937a-e30522ffc0df\") " pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.433142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433056    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433079    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-run-netns\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433100    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-log-socket\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433124    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-cni-netd\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433147    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f311012a-e71c-45af-b64d-83e2245f52ad-ovnkube-script-lib\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433185    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa938a91-1541-485a-b88f-151697650d2a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433219    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn276\" (UniqueName: \"kubernetes.io/projected/cfc5e2be-0dac-4a19-9c82-f70065c9a9b4-kube-api-access-jn276\") pod \"node-resolver-dwlgl\" (UID: \"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4\") " pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433304    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g2f2\" (UniqueName: \"kubernetes.io/projected/87136fc5-90a6-4728-9410-a51d6050e497-kube-api-access-7g2f2\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433336    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-run-systemd\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433370    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f311012a-e71c-45af-b64d-83e2245f52ad-env-overrides\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433392    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa938a91-1541-485a-b88f-151697650d2a-system-cni-dir\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433416    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-device-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433445    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-sysctl-d\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433467    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-host\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433489    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-cni-dir\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433511    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-etc-kubernetes\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.433922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433532    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-kubelet\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433560    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cfc5e2be-0dac-4a19-9c82-f70065c9a9b4-hosts-file\") pod \"node-resolver-dwlgl\" (UID: \"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4\") " pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433580    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-var-lib-kubelet\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433603    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-system-cni-dir\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433620    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-run-ovn\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433641    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvrd8\" (UniqueName: \"kubernetes.io/projected/ff0b971f-0f3c-45dd-8234-c68a65df7eae-kube-api-access-xvrd8\") pod \"iptables-alerter-rlv9n\" (UID: \"ff0b971f-0f3c-45dd-8234-c68a65df7eae\") " pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433658    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-systemd-units\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433693    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa938a91-1541-485a-b88f-151697650d2a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433744    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa938a91-1541-485a-b88f-151697650d2a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433771    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-sys\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433793    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-run-k8s-cni-cncf-io\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433819    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-registration-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433841    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-lib-modules\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433866    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-run-netns\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433889    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-var-lib-cni-bin\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.434758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.433913    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-slash\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.465887 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.465550    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 21:05:25 +0000 UTC" deadline="2028-06-13 16:06:24.518058498 +0000 UTC"
Sep 10 21:10:25.465887 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.465580    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15402h55m59.052482143s"
Sep 10 21:10:25.522112 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.522081    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 21:10:25.528797 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.528764    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:25.534609 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534576    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-systemd-units\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.534609 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534612    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa938a91-1541-485a-b88f-151697650d2a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.534609 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534636    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa938a91-1541-485a-b88f-151697650d2a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534660    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-sys\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534684    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-run-k8s-cni-cncf-io\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534690    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-systemd-units\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534707    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-registration-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534729    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-lib-modules\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534756    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-sys\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534797    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-run-k8s-cni-cncf-io\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534814    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-registration-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534848    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa938a91-1541-485a-b88f-151697650d2a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534864    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-run-netns\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.534907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534889    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-var-lib-cni-bin\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534929    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-run-netns\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534939    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-lib-modules\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534948    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-slash\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534981    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-slash\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534984    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa938a91-1541-485a-b88f-151697650d2a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.534930    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-var-lib-cni-bin\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535015    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wzp\" (UniqueName: \"kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp\") pod \"network-check-target-4zmfv\" (UID: \"492738f6-5f08-4101-837f-87cf59c8e113\") " pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535045    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-socket-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535070    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhjbq\" (UniqueName: \"kubernetes.io/projected/3fbc2efe-e578-40d2-8897-ddcb48ef5588-kube-api-access-nhjbq\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535091    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-run\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535112    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e94852a3-19d2-4766-87ac-187802031bf8-etc-tuned\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535135    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1f372f51-ce6b-46c2-9438-df2f95851903-dbus\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535157    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-var-lib-openvswitch\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535177    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa938a91-1541-485a-b88f-151697650d2a-cnibin\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535197    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-socket-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535200    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-sysctl-conf\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.535365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535219    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa938a91-1541-485a-b88f-151697650d2a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535253    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-socket-dir-parent\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535303    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-var-lib-kubelet\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535304    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1f372f51-ce6b-46c2-9438-df2f95851903-dbus\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535336    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-var-lib-openvswitch\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535341    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-sysctl-conf\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535349    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-var-lib-kubelet\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535357    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-socket-dir-parent\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535370    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff0b971f-0f3c-45dd-8234-c68a65df7eae-iptables-alerter-script\") pod \"iptables-alerter-rlv9n\" (UID: \"ff0b971f-0f3c-45dd-8234-c68a65df7eae\") " pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535367    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa938a91-1541-485a-b88f-151697650d2a-cnibin\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535394    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-kubelet-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535399    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-run\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535414    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-etc-selinux\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535427    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-kubelet-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535430    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-run-multus-certs\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535459    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-run-multus-certs\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535467    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-etc-selinux\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.536093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535474    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1f372f51-ce6b-46c2-9438-df2f95851903-kubelet-config\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535496    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0e6d31a2-19e9-4970-937a-e30522ffc0df-serviceca\") pod \"node-ca-svw8v\" (UID: \"0e6d31a2-19e9-4970-937a-e30522ffc0df\") " pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535511    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-kubernetes\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535513    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1f372f51-ce6b-46c2-9438-df2f95851903-kubelet-config\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535525    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19e148c2-b280-48d0-ac44-7246e35a71ea-cni-binary-copy\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535547    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-cni-bin\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535568    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f311012a-e71c-45af-b64d-83e2245f52ad-ovnkube-config\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535587    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-sysconfig\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535609    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3160b92d-6922-4d94-a5ed-b92338837c2f-konnectivity-ca\") pod \"konnectivity-agent-vttxv\" (UID: \"3160b92d-6922-4d94-a5ed-b92338837c2f\") " pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535644    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535669    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfc5e2be-0dac-4a19-9c82-f70065c9a9b4-tmp-dir\") pod \"node-resolver-dwlgl\" (UID: \"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4\") " pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535692    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535716    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-sys-fs\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535737    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-modprobe-d\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535764    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-daemon-config\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535763    2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535817    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff0b971f-0f3c-45dd-8234-c68a65df7eae-iptables-alerter-script\") pod \"iptables-alerter-rlv9n\" (UID: \"ff0b971f-0f3c-45dd-8234-c68a65df7eae\") " pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535825    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-etc-openvswitch\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.536844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535647    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-cni-bin\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535895    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-sysconfig\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535619    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-kubernetes\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535933    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535962    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0e6d31a2-19e9-4970-937a-e30522ffc0df-serviceca\") pod \"node-ca-svw8v\" (UID: \"0e6d31a2-19e9-4970-937a-e30522ffc0df\") " pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535969    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-sys-fs\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.536049    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536058    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-modprobe-d\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.536133    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs podName:87136fc5-90a6-4728-9410-a51d6050e497 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:26.036090346 +0000 UTC m=+2.972263659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs") pod "network-metrics-daemon-vqprq" (UID: "87136fc5-90a6-4728-9410-a51d6050e497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536238    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfc5e2be-0dac-4a19-9c82-f70065c9a9b4-tmp-dir\") pod \"node-resolver-dwlgl\" (UID: \"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4\") " pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536259    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19e148c2-b280-48d0-ac44-7246e35a71ea-cni-binary-copy\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536300    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa938a91-1541-485a-b88f-151697650d2a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.535792    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-etc-openvswitch\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536404    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff0b971f-0f3c-45dd-8234-c68a65df7eae-host-slash\") pod \"iptables-alerter-rlv9n\" (UID: \"ff0b971f-0f3c-45dd-8234-c68a65df7eae\") " pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536422    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e94852a3-19d2-4766-87ac-187802031bf8-tmp\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536466    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-hostroot\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536481    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-conf-dir\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.537724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536484    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3160b92d-6922-4d94-a5ed-b92338837c2f-konnectivity-ca\") pod \"konnectivity-agent-vttxv\" (UID: \"3160b92d-6922-4d94-a5ed-b92338837c2f\") " pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536496    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-node-log\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536516    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w49d\" (UniqueName: \"kubernetes.io/projected/f311012a-e71c-45af-b64d-83e2245f52ad-kube-api-access-8w49d\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536533    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mn5p\" (UniqueName: \"kubernetes.io/projected/fa938a91-1541-485a-b88f-151697650d2a-kube-api-access-5mn5p\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536547    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-systemd\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536552    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-daemon-config\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536563    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s76k9\" (UniqueName: \"kubernetes.io/projected/e94852a3-19d2-4766-87ac-187802031bf8-kube-api-access-s76k9\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536573    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff0b971f-0f3c-45dd-8234-c68a65df7eae-host-slash\") pod \"iptables-alerter-rlv9n\" (UID: \"ff0b971f-0f3c-45dd-8234-c68a65df7eae\") " pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536588    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3160b92d-6922-4d94-a5ed-b92338837c2f-agent-certs\") pod \"konnectivity-agent-vttxv\" (UID: \"3160b92d-6922-4d94-a5ed-b92338837c2f\") " pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536619    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-run-openvswitch\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536646    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536658    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-node-log\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536543    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-hostroot\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536695    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536695    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-systemd\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536714    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-cnibin\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536747    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-run-openvswitch\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536757    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-var-lib-cni-multus\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.538746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536782    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbxsr\" (UniqueName: \"kubernetes.io/projected/19e148c2-b280-48d0-ac44-7246e35a71ea-kube-api-access-tbxsr\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536784    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-conf-dir\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536789    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f311012a-e71c-45af-b64d-83e2245f52ad-ovnkube-config\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536806    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f311012a-e71c-45af-b64d-83e2245f52ad-ovn-node-metrics-cert\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536831    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e6d31a2-19e9-4970-937a-e30522ffc0df-host\") pod \"node-ca-svw8v\" (UID: \"0e6d31a2-19e9-4970-937a-e30522ffc0df\") " pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536829    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-host-var-lib-cni-multus\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536842    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-cnibin\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536912    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e6d31a2-19e9-4970-937a-e30522ffc0df-host\") pod \"node-ca-svw8v\" (UID: \"0e6d31a2-19e9-4970-937a-e30522ffc0df\") " pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.536992    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m8qs\" (UniqueName: \"kubernetes.io/projected/0e6d31a2-19e9-4970-937a-e30522ffc0df-kube-api-access-7m8qs\") pod \"node-ca-svw8v\" (UID: \"0e6d31a2-19e9-4970-937a-e30522ffc0df\") " pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537025    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537053    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-run-netns\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537077    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-log-socket\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537101    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-cni-netd\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537128    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f311012a-e71c-45af-b64d-83e2245f52ad-ovnkube-script-lib\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.537136    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537153    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa938a91-1541-485a-b88f-151697650d2a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.537195    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret podName:1f372f51-ce6b-46c2-9438-df2f95851903 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:26.037178056 +0000 UTC m=+2.973351365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret") pod "global-pull-secret-syncer-z8k59" (UID: "1f372f51-ce6b-46c2-9438-df2f95851903") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:25.539623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537216    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn276\" (UniqueName: \"kubernetes.io/projected/cfc5e2be-0dac-4a19-9c82-f70065c9a9b4-kube-api-access-jn276\") pod \"node-resolver-dwlgl\" (UID: \"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4\") " pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537247    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g2f2\" (UniqueName: \"kubernetes.io/projected/87136fc5-90a6-4728-9410-a51d6050e497-kube-api-access-7g2f2\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537289    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-run-systemd\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537315    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f311012a-e71c-45af-b64d-83e2245f52ad-env-overrides\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537342    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa938a91-1541-485a-b88f-151697650d2a-system-cni-dir\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537366    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-device-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537418    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3fbc2efe-e578-40d2-8897-ddcb48ef5588-device-dir\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537584    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-log-socket\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537598    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-sysctl-d\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537628    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-host\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537645    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-run-netns\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537644    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa938a91-1541-485a-b88f-151697650d2a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537673    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-cni-dir\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537690    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-run-systemd\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537717    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-host\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537735    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-multus-cni-dir\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537806    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-etc-sysctl-d\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.540444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537848    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa938a91-1541-485a-b88f-151697650d2a-system-cni-dir\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537884    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-cni-netd\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537927    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-etc-kubernetes\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537949    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-kubelet\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537968    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cfc5e2be-0dac-4a19-9c82-f70065c9a9b4-hosts-file\") pod \"node-resolver-dwlgl\" (UID: \"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4\") " pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.537988    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-var-lib-kubelet\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538010    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-system-cni-dir\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538016    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-host-kubelet\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538077    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cfc5e2be-0dac-4a19-9c82-f70065c9a9b4-hosts-file\") pod \"node-resolver-dwlgl\" (UID: \"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4\") " pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538084    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f311012a-e71c-45af-b64d-83e2245f52ad-env-overrides\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538078    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-run-ovn\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538109    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-etc-kubernetes\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538035    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f311012a-e71c-45af-b64d-83e2245f52ad-run-ovn\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538133    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19e148c2-b280-48d0-ac44-7246e35a71ea-system-cni-dir\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538144    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvrd8\" (UniqueName: \"kubernetes.io/projected/ff0b971f-0f3c-45dd-8234-c68a65df7eae-kube-api-access-xvrd8\") pod \"iptables-alerter-rlv9n\" (UID: \"ff0b971f-0f3c-45dd-8234-c68a65df7eae\") " pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538140    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f311012a-e71c-45af-b64d-83e2245f52ad-ovnkube-script-lib\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.538147    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e94852a3-19d2-4766-87ac-187802031bf8-var-lib-kubelet\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.540053    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e94852a3-19d2-4766-87ac-187802031bf8-tmp\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.541145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.540243    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f311012a-e71c-45af-b64d-83e2245f52ad-ovn-node-metrics-cert\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.541917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.540377    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3160b92d-6922-4d94-a5ed-b92338837c2f-agent-certs\") pod \"konnectivity-agent-vttxv\" (UID: \"3160b92d-6922-4d94-a5ed-b92338837c2f\") " pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:25.541917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.540838    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e94852a3-19d2-4766-87ac-187802031bf8-etc-tuned\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.542154 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.542137    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:25.542195 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.542160    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:25.542195 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.542174    2573 projected.go:196] Error preparing data for projected volume kube-api-access-h4wzp for pod openshift-network-diagnostics/network-check-target-4zmfv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:25.542283 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:25.542237    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp podName:492738f6-5f08-4101-837f-87cf59c8e113 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:26.042221595 +0000 UTC m=+2.978394888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h4wzp" (UniqueName: "kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp") pod "network-check-target-4zmfv" (UID: "492738f6-5f08-4101-837f-87cf59c8e113") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:25.544748 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.544724    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhjbq\" (UniqueName: \"kubernetes.io/projected/3fbc2efe-e578-40d2-8897-ddcb48ef5588-kube-api-access-nhjbq\") pod \"aws-ebs-csi-driver-node-48lxf\" (UID: \"3fbc2efe-e578-40d2-8897-ddcb48ef5588\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.547245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.547193    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mn5p\" (UniqueName: \"kubernetes.io/projected/fa938a91-1541-485a-b88f-151697650d2a-kube-api-access-5mn5p\") pod \"multus-additional-cni-plugins-fc4cj\" (UID: \"fa938a91-1541-485a-b88f-151697650d2a\") " pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.550218 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.549877    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbxsr\" (UniqueName: \"kubernetes.io/projected/19e148c2-b280-48d0-ac44-7246e35a71ea-kube-api-access-tbxsr\") pod \"multus-wh98p\" (UID: \"19e148c2-b280-48d0-ac44-7246e35a71ea\") " pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.550218 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.550018    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w49d\" (UniqueName: \"kubernetes.io/projected/f311012a-e71c-45af-b64d-83e2245f52ad-kube-api-access-8w49d\") pod \"ovnkube-node-bp6vk\" (UID: \"f311012a-e71c-45af-b64d-83e2245f52ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.550218 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.550177    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m8qs\" (UniqueName: \"kubernetes.io/projected/0e6d31a2-19e9-4970-937a-e30522ffc0df-kube-api-access-7m8qs\") pod \"node-ca-svw8v\" (UID: \"0e6d31a2-19e9-4970-937a-e30522ffc0df\") " pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.550422 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.550296    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s76k9\" (UniqueName: \"kubernetes.io/projected/e94852a3-19d2-4766-87ac-187802031bf8-kube-api-access-s76k9\") pod \"tuned-82gdq\" (UID: \"e94852a3-19d2-4766-87ac-187802031bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.550564 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.550506    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn276\" (UniqueName: \"kubernetes.io/projected/cfc5e2be-0dac-4a19-9c82-f70065c9a9b4-kube-api-access-jn276\") pod \"node-resolver-dwlgl\" (UID: \"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4\") " pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.552161 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.552142    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g2f2\" (UniqueName: \"kubernetes.io/projected/87136fc5-90a6-4728-9410-a51d6050e497-kube-api-access-7g2f2\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:25.552333 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.552317    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvrd8\" (UniqueName: \"kubernetes.io/projected/ff0b971f-0f3c-45dd-8234-c68a65df7eae-kube-api-access-xvrd8\") pod \"iptables-alerter-rlv9n\" (UID: \"ff0b971f-0f3c-45dd-8234-c68a65df7eae\") " pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:25.675612 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.675532    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal" event={"ID":"e53f32f7691c310cd328965d8dd7641c","Type":"ContainerStarted","Data":"64f96da4a18b80a1d9f8c27df74a209672b13eef95bc248317f8c92e4ac4c078"}
Sep 10 21:10:25.721694 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.721656    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dwlgl"
Sep 10 21:10:25.727765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.727737    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-svw8v"
Sep 10 21:10:25.730159 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.730138    2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:25.738988 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.738954    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:25.745976 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.745957    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf"
Sep 10 21:10:25.753402 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.753384    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-82gdq"
Sep 10 21:10:25.762069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.762048    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fc4cj"
Sep 10 21:10:25.770143 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.770115    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:25.778301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.778280    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wh98p"
Sep 10 21:10:25.786033 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:25.786014    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rlv9n"
Sep 10 21:10:27.044062 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.043977    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:27.044062 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.044023    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:27.044278 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:27.044137    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:27.044278 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:27.044199    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs podName:87136fc5-90a6-4728-9410-a51d6050e497 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:28.044184779 +0000 UTC m=+3.977851576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs") pod "network-metrics-daemon-vqprq" (UID: "87136fc5-90a6-4728-9410-a51d6050e497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:27.044278 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:27.044137    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:27.044278 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:27.044268    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret podName:1f372f51-ce6b-46c2-9438-df2f95851903 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:28.044256322 +0000 UTC m=+3.977923129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret") pod "global-pull-secret-syncer-z8k59" (UID: "1f372f51-ce6b-46c2-9438-df2f95851903") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:27.145117 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.145077    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wzp\" (UniqueName: \"kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp\") pod \"network-check-target-4zmfv\" (UID: \"492738f6-5f08-4101-837f-87cf59c8e113\") " pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:27.145308 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:27.145261    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:27.145308 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:27.145285    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:27.145308 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:27.145296    2573 projected.go:196] Error preparing data for projected volume kube-api-access-h4wzp for pod openshift-network-diagnostics/network-check-target-4zmfv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:27.145460 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:27.145358    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp podName:492738f6-5f08-4101-837f-87cf59c8e113 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:28.145343237 +0000 UTC m=+4.079010032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4wzp" (UniqueName: "kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp") pod "network-check-target-4zmfv" (UID: "492738f6-5f08-4101-837f-87cf59c8e113") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:27.468408 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.468352    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 21:05:25 +0000 UTC" deadline="2028-06-12 08:42:33.940949068 +0000 UTC"
Sep 10 21:10:27.468408 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.468401    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15371h32m6.472550397s"
Sep 10 21:10:27.634716 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:27.634552    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfc5e2be_0dac_4a19_9c82_f70065c9a9b4.slice/crio-e943d67d0730a1cb3fdedf38521a4d101925bcfd801d204516a4b11f2355713c WatchSource:0}: Error finding container e943d67d0730a1cb3fdedf38521a4d101925bcfd801d204516a4b11f2355713c: Status 404 returned error can't find the container with id e943d67d0730a1cb3fdedf38521a4d101925bcfd801d204516a4b11f2355713c
Sep 10 21:10:27.648948 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:27.648828    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e6d31a2_19e9_4970_937a_e30522ffc0df.slice/crio-979a7cbc2303f2c44a4ea967b557ed1a37cb8d4a745cf172a673d880282655aa WatchSource:0}: Error finding container 979a7cbc2303f2c44a4ea967b557ed1a37cb8d4a745cf172a673d880282655aa: Status 404 returned error can't find the container with id 979a7cbc2303f2c44a4ea967b557ed1a37cb8d4a745cf172a673d880282655aa
Sep 10 21:10:27.654213 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:27.654190    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19e148c2_b280_48d0_ac44_7246e35a71ea.slice/crio-e1261811f900edbe348f8ee84b26d78527b5ba03e8b4beb8928f072723770d84 WatchSource:0}: Error finding container e1261811f900edbe348f8ee84b26d78527b5ba03e8b4beb8928f072723770d84: Status 404 returned error can't find the container with id e1261811f900edbe348f8ee84b26d78527b5ba03e8b4beb8928f072723770d84
Sep 10 21:10:27.655014 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:27.654969    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf311012a_e71c_45af_b64d_83e2245f52ad.slice/crio-ca9b258a15ee9118b1ade4e54d0ee16820008901cf9d5424c973cd6164ff2346 WatchSource:0}: Error finding container ca9b258a15ee9118b1ade4e54d0ee16820008901cf9d5424c973cd6164ff2346: Status 404 returned error can't find the container with id ca9b258a15ee9118b1ade4e54d0ee16820008901cf9d5424c973cd6164ff2346
Sep 10 21:10:27.656100 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:27.656074    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0b971f_0f3c_45dd_8234_c68a65df7eae.slice/crio-42aac3aa89b105d7bdeb7063a8374157a873ba500c66d20e9cecd02689795da4 WatchSource:0}: Error finding container 42aac3aa89b105d7bdeb7063a8374157a873ba500c66d20e9cecd02689795da4: Status 404 returned error can't find the container with id 42aac3aa89b105d7bdeb7063a8374157a873ba500c66d20e9cecd02689795da4
Sep 10 21:10:27.657367 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:27.657342    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa938a91_1541_485a_b88f_151697650d2a.slice/crio-b686a55da391efe0e1abeecd2ed090ea2e93731da073351ee7cc05f4af4cf361 WatchSource:0}: Error finding container b686a55da391efe0e1abeecd2ed090ea2e93731da073351ee7cc05f4af4cf361: Status 404 returned error can't find the container with id b686a55da391efe0e1abeecd2ed090ea2e93731da073351ee7cc05f4af4cf361
Sep 10 21:10:27.659105 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:27.659023    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fbc2efe_e578_40d2_8897_ddcb48ef5588.slice/crio-96ca6247ae27226e89ce0077df7bbfe52afcb607a4e4b2529bd412fc73f98c11 WatchSource:0}: Error finding container 96ca6247ae27226e89ce0077df7bbfe52afcb607a4e4b2529bd412fc73f98c11: Status 404 returned error can't find the container with id 96ca6247ae27226e89ce0077df7bbfe52afcb607a4e4b2529bd412fc73f98c11
Sep 10 21:10:27.659952 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:27.659931    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3160b92d_6922_4d94_a5ed_b92338837c2f.slice/crio-7d13898362e313002c207979943ee8a088b11b91f850972beb8bd2949bbce2d3 WatchSource:0}: Error finding container 7d13898362e313002c207979943ee8a088b11b91f850972beb8bd2949bbce2d3: Status 404 returned error can't find the container with id 7d13898362e313002c207979943ee8a088b11b91f850972beb8bd2949bbce2d3
Sep 10 21:10:27.662164 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:10:27.661365    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94852a3_19d2_4766_87ac_187802031bf8.slice/crio-25a8530c657a9e608e93a39275e9b855bad5d3f5c09d6e21d7da621f948185ca WatchSource:0}: Error finding container 25a8530c657a9e608e93a39275e9b855bad5d3f5c09d6e21d7da621f948185ca: Status 404 returned error can't find the container with id 25a8530c657a9e608e93a39275e9b855bad5d3f5c09d6e21d7da621f948185ca
Sep 10 21:10:27.672652 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.672626    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:27.672742 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.672663    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:27.672742 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:27.672735    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:27.672887 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:27.672822    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:27.681200 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.681175    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wh98p" event={"ID":"19e148c2-b280-48d0-ac44-7246e35a71ea","Type":"ContainerStarted","Data":"e1261811f900edbe348f8ee84b26d78527b5ba03e8b4beb8928f072723770d84"}
Sep 10 21:10:27.682325 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.682302    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-svw8v" event={"ID":"0e6d31a2-19e9-4970-937a-e30522ffc0df","Type":"ContainerStarted","Data":"979a7cbc2303f2c44a4ea967b557ed1a37cb8d4a745cf172a673d880282655aa"}
Sep 10 21:10:27.683580 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.683553    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-82gdq" event={"ID":"e94852a3-19d2-4766-87ac-187802031bf8","Type":"ContainerStarted","Data":"25a8530c657a9e608e93a39275e9b855bad5d3f5c09d6e21d7da621f948185ca"}
Sep 10 21:10:27.684634 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.684600    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vttxv" event={"ID":"3160b92d-6922-4d94-a5ed-b92338837c2f","Type":"ContainerStarted","Data":"7d13898362e313002c207979943ee8a088b11b91f850972beb8bd2949bbce2d3"}
Sep 10 21:10:27.685951 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.685932    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf" event={"ID":"3fbc2efe-e578-40d2-8897-ddcb48ef5588","Type":"ContainerStarted","Data":"96ca6247ae27226e89ce0077df7bbfe52afcb607a4e4b2529bd412fc73f98c11"}
Sep 10 21:10:27.687446 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.687428    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fc4cj" event={"ID":"fa938a91-1541-485a-b88f-151697650d2a","Type":"ContainerStarted","Data":"b686a55da391efe0e1abeecd2ed090ea2e93731da073351ee7cc05f4af4cf361"}
Sep 10 21:10:27.688911 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.688888    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rlv9n" event={"ID":"ff0b971f-0f3c-45dd-8234-c68a65df7eae","Type":"ContainerStarted","Data":"42aac3aa89b105d7bdeb7063a8374157a873ba500c66d20e9cecd02689795da4"}
Sep 10 21:10:27.691927 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.691901    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk" event={"ID":"f311012a-e71c-45af-b64d-83e2245f52ad","Type":"ContainerStarted","Data":"ca9b258a15ee9118b1ade4e54d0ee16820008901cf9d5424c973cd6164ff2346"}
Sep 10 21:10:27.698933 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:27.698907    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dwlgl" event={"ID":"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4","Type":"ContainerStarted","Data":"e943d67d0730a1cb3fdedf38521a4d101925bcfd801d204516a4b11f2355713c"}
Sep 10 21:10:28.051236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:28.051151    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:28.051236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:28.051218    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:28.051455 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:28.051318    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:28.051455 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:28.051367    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:28.051455 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:28.051384    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs podName:87136fc5-90a6-4728-9410-a51d6050e497 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:30.051369583 +0000 UTC m=+5.985036376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs") pod "network-metrics-daemon-vqprq" (UID: "87136fc5-90a6-4728-9410-a51d6050e497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:28.051455 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:28.051425    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret podName:1f372f51-ce6b-46c2-9438-df2f95851903 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:30.051408459 +0000 UTC m=+5.985075273 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret") pod "global-pull-secret-syncer-z8k59" (UID: "1f372f51-ce6b-46c2-9438-df2f95851903") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:28.152748 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:28.152708    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wzp\" (UniqueName: \"kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp\") pod \"network-check-target-4zmfv\" (UID: \"492738f6-5f08-4101-837f-87cf59c8e113\") " pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:28.152983 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:28.152963    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:28.153052 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:28.152992    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:28.153052 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:28.153005    2573 projected.go:196] Error preparing data for projected volume kube-api-access-h4wzp for pod openshift-network-diagnostics/network-check-target-4zmfv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:28.153243 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:28.153066    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp podName:492738f6-5f08-4101-837f-87cf59c8e113 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:30.153048473 +0000 UTC m=+6.086715267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4wzp" (UniqueName: "kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp") pod "network-check-target-4zmfv" (UID: "492738f6-5f08-4101-837f-87cf59c8e113") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:28.675007 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:28.674971    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:28.675805 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:28.675123    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:28.719788 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:28.719733    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal" event={"ID":"bcd962acf554a8e9018c7c906807b563","Type":"ContainerStarted","Data":"153a864dcaa5b420346ee5abf58669c431792e0a88cf220952e9d8ab7a716c52"}
Sep 10 21:10:28.736390 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:28.732519    2573 generic.go:356] "Generic (PLEG): container finished" podID="e53f32f7691c310cd328965d8dd7641c" containerID="0a74833fa8c6c2eb0f2bccf26e78b61122fd7bc9f6b688de5474727b559761ea" exitCode=0
Sep 10 21:10:28.736390 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:28.732572    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal" event={"ID":"e53f32f7691c310cd328965d8dd7641c","Type":"ContainerDied","Data":"0a74833fa8c6c2eb0f2bccf26e78b61122fd7bc9f6b688de5474727b559761ea"}
Sep 10 21:10:28.752268 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:28.752208    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-74.ec2.internal" podStartSLOduration=3.752189883 podStartE2EDuration="3.752189883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:28.736637915 +0000 UTC m=+4.670304735" watchObservedRunningTime="2026-09-10 21:10:28.752189883 +0000 UTC m=+4.685856699"
Sep 10 21:10:29.673481 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:29.672955    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:29.673481 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:29.672986    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:29.673481 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:29.673098    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:29.673481 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:29.673226    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:29.744645 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:29.744607    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal" event={"ID":"e53f32f7691c310cd328965d8dd7641c","Type":"ContainerStarted","Data":"cc4a4bad194c95efd9e39dde15034483c33107fce64742ad975f077cebad5c05"}
Sep 10 21:10:30.070045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:30.069945    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:30.070234 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:30.070051    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:30.070304 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:30.070272    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:30.070360 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:30.070337    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret podName:1f372f51-ce6b-46c2-9438-df2f95851903 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:34.070317884 +0000 UTC m=+10.003984682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret") pod "global-pull-secret-syncer-z8k59" (UID: "1f372f51-ce6b-46c2-9438-df2f95851903") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:30.070685 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:30.070530    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:30.070685 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:30.070605    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs podName:87136fc5-90a6-4728-9410-a51d6050e497 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:34.070586273 +0000 UTC m=+10.004253082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs") pod "network-metrics-daemon-vqprq" (UID: "87136fc5-90a6-4728-9410-a51d6050e497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:30.171322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:30.171282    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wzp\" (UniqueName: \"kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp\") pod \"network-check-target-4zmfv\" (UID: \"492738f6-5f08-4101-837f-87cf59c8e113\") " pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:30.171641 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:30.171589    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:30.171641 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:30.171614    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:30.171641 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:30.171627    2573 projected.go:196] Error preparing data for projected volume kube-api-access-h4wzp for pod openshift-network-diagnostics/network-check-target-4zmfv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:30.171906 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:30.171744    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp podName:492738f6-5f08-4101-837f-87cf59c8e113 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:34.171722486 +0000 UTC m=+10.105389282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4wzp" (UniqueName: "kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp") pod "network-check-target-4zmfv" (UID: "492738f6-5f08-4101-837f-87cf59c8e113") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:30.683910 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:30.683881    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:30.684106 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:30.684014    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:31.673426 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:31.673384    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:31.673905 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:31.673520    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:31.674012 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:31.673991    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:31.674122 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:31.674100    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:32.673614 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:32.673578    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:32.674102 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:32.673723    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:33.673516 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:33.673480    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:33.673708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:33.673536    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:33.674048 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:33.673723    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:33.674048 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:33.673862    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:34.105601 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:34.105557    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:34.105799 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:34.105661    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:34.105799 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:34.105747    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:34.105912 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:34.105748    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:34.105912 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:34.105829    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs podName:87136fc5-90a6-4728-9410-a51d6050e497 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:42.105809175 +0000 UTC m=+18.039475993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs") pod "network-metrics-daemon-vqprq" (UID: "87136fc5-90a6-4728-9410-a51d6050e497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:34.105912 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:34.105907    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret podName:1f372f51-ce6b-46c2-9438-df2f95851903 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:42.105875412 +0000 UTC m=+18.039542219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret") pod "global-pull-secret-syncer-z8k59" (UID: "1f372f51-ce6b-46c2-9438-df2f95851903") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:34.207463 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:34.206759    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wzp\" (UniqueName: \"kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp\") pod \"network-check-target-4zmfv\" (UID: \"492738f6-5f08-4101-837f-87cf59c8e113\") " pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:34.207463 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:34.207009    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:34.207463 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:34.207029    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:34.207463 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:34.207042    2573 projected.go:196] Error preparing data for projected volume kube-api-access-h4wzp for pod openshift-network-diagnostics/network-check-target-4zmfv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:34.207463 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:34.207108    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp podName:492738f6-5f08-4101-837f-87cf59c8e113 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:42.207089725 +0000 UTC m=+18.140756520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4wzp" (UniqueName: "kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp") pod "network-check-target-4zmfv" (UID: "492738f6-5f08-4101-837f-87cf59c8e113") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:34.674488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:34.674449    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:34.674917 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:34.674588    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:35.674461 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:35.673341    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:35.674461 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:35.673475    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:35.674461 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:35.673891    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:35.674461 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:35.673988    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:36.677170 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:36.677135    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:36.677626 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:36.677266    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:37.673266 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:37.673226    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:37.673458 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:37.673232    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:37.673458 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:37.673359    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:37.673458 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:37.673437    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:38.673309 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:38.673275    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:38.673740 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:38.673390    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:39.673098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:39.673062    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:39.673302 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:39.673067    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:39.673302 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:39.673173    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:39.673302 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:39.673261    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:40.673369 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:40.673333    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:40.673852 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:40.673466    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:41.673268 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:41.673234    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:41.673492 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:41.673367    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:41.673492 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:41.673429    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:41.673866 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:41.673552    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:42.172926 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:42.172710    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:42.173111 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:42.172971    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:42.173111 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:42.172876    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:42.173111 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:42.173061    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs podName:87136fc5-90a6-4728-9410-a51d6050e497 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:58.173039441 +0000 UTC m=+34.106706249 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs") pod "network-metrics-daemon-vqprq" (UID: "87136fc5-90a6-4728-9410-a51d6050e497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:42.173111 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:42.173074    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:42.173321 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:42.173122    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret podName:1f372f51-ce6b-46c2-9438-df2f95851903 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:58.173109573 +0000 UTC m=+34.106776366 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret") pod "global-pull-secret-syncer-z8k59" (UID: "1f372f51-ce6b-46c2-9438-df2f95851903") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:42.273370 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:42.273333    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wzp\" (UniqueName: \"kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp\") pod \"network-check-target-4zmfv\" (UID: \"492738f6-5f08-4101-837f-87cf59c8e113\") " pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:42.273556 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:42.273488    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:42.273556 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:42.273507    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:42.273556 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:42.273519    2573 projected.go:196] Error preparing data for projected volume kube-api-access-h4wzp for pod openshift-network-diagnostics/network-check-target-4zmfv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:42.273670 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:42.273575    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp podName:492738f6-5f08-4101-837f-87cf59c8e113 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:58.273559868 +0000 UTC m=+34.207226682 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4wzp" (UniqueName: "kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp") pod "network-check-target-4zmfv" (UID: "492738f6-5f08-4101-837f-87cf59c8e113") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:42.673489 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:42.673444    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:42.673922 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:42.673578    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:43.672880 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:43.672840    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:43.672880 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:43.672865    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:43.673162 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:43.672971    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:43.673162 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:43.673118    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:44.674299 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:44.674260    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:44.674743 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:44.674384    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:45.672611 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.672511    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:45.672759 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:45.672615    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:45.672759 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.672652    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:45.672759 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:45.672694    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:45.781901 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.781607    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-82gdq" event={"ID":"e94852a3-19d2-4766-87ac-187802031bf8","Type":"ContainerStarted","Data":"1e373cadc361a2a930e0b35b2e978ac488e511d237c5d5ed73491d2aacacbcff"}
Sep 10 21:10:45.783607 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.783573    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vttxv" event={"ID":"3160b92d-6922-4d94-a5ed-b92338837c2f","Type":"ContainerStarted","Data":"29be85798268ba423a43cd105bc48bb351ea0fa5e6798311c1b0384e893ee0db"}
Sep 10 21:10:45.785385 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.785347    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf" event={"ID":"3fbc2efe-e578-40d2-8897-ddcb48ef5588","Type":"ContainerStarted","Data":"4c373ee22cacc5820ad2f96beadedbfa38075c6476c05cd00142322eeab66772"}
Sep 10 21:10:45.787074 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.787053    2573 generic.go:356] "Generic (PLEG): container finished" podID="fa938a91-1541-485a-b88f-151697650d2a" containerID="6259cb7c592d5620d9b6b6634a647c9bae0ff3fc77f70b2f93a0df08190f4652" exitCode=0
Sep 10 21:10:45.787174 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.787146    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fc4cj" event={"ID":"fa938a91-1541-485a-b88f-151697650d2a","Type":"ContainerDied","Data":"6259cb7c592d5620d9b6b6634a647c9bae0ff3fc77f70b2f93a0df08190f4652"}
Sep 10 21:10:45.789759 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.789735    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk" event={"ID":"f311012a-e71c-45af-b64d-83e2245f52ad","Type":"ContainerStarted","Data":"91ef78b3b8c6a0ba0bf7e25a7915e473d3c2cfc6e076e150a34eadeb217a1f6e"}
Sep 10 21:10:45.789933 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.789862    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk" event={"ID":"f311012a-e71c-45af-b64d-83e2245f52ad","Type":"ContainerStarted","Data":"14d9f12a87cb2af295f8f3a8ba0b00d6acae6655800143b6a9d45f8f60f94ffc"}
Sep 10 21:10:45.791535 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.791493    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dwlgl" event={"ID":"cfc5e2be-0dac-4a19-9c82-f70065c9a9b4","Type":"ContainerStarted","Data":"82f975691c70265a8711f27801ed699ba782f8ea3ffa6e8b1ea9b77a87f37655"}
Sep 10 21:10:45.793508 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.793483    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wh98p" event={"ID":"19e148c2-b280-48d0-ac44-7246e35a71ea","Type":"ContainerStarted","Data":"f70418f28867db811be9be84196e44419d4101fa11f008136591fdfcd8071a42"}
Sep 10 21:10:45.795337 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.795313    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-svw8v" event={"ID":"0e6d31a2-19e9-4970-937a-e30522ffc0df","Type":"ContainerStarted","Data":"fa30df00331447d315d598066a24b6efa6f0aca94f6b3a777bad96a00aa8bf41"}
Sep 10 21:10:45.800899 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.800853    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-74.ec2.internal" podStartSLOduration=20.800767819 podStartE2EDuration="20.800767819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:29.761294634 +0000 UTC m=+5.694961451" watchObservedRunningTime="2026-09-10 21:10:45.800767819 +0000 UTC m=+21.734434635"
Sep 10 21:10:45.815813 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.815748    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-82gdq" podStartSLOduration=4.054739831 podStartE2EDuration="21.815727988s" totalImagesPullingTime="17.760988157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:45.800176871 +0000 UTC m=+21.733843686" watchObservedRunningTime="2026-09-10 21:10:45.815727988 +0000 UTC m=+21.749394804"
Sep 10 21:10:45.833601 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.833543    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dwlgl" podStartSLOduration=4.084833463 podStartE2EDuration="21.833521656s" totalImagesPullingTime="17.748688193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:45.815956407 +0000 UTC m=+21.749623222" watchObservedRunningTime="2026-09-10 21:10:45.833521656 +0000 UTC m=+21.767188474"
Sep 10 21:10:45.833862 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.833829    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wh98p" podStartSLOduration=3.941239439 podStartE2EDuration="21.83381765s" totalImagesPullingTime="17.892578211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:45.833391329 +0000 UTC m=+21.767058143" watchObservedRunningTime="2026-09-10 21:10:45.83381765 +0000 UTC m=+21.767484468"
Sep 10 21:10:45.855830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.855754    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-svw8v" podStartSLOduration=4.146509631 podStartE2EDuration="21.855732985s" totalImagesPullingTime="17.709223354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:45.855489305 +0000 UTC m=+21.789156119" watchObservedRunningTime="2026-09-10 21:10:45.855732985 +0000 UTC m=+21.789399802"
Sep 10 21:10:45.901054 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:45.900982    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vttxv" podStartSLOduration=8.363742936 podStartE2EDuration="21.900959113s" totalImagesPullingTime="13.537216177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:45.900445828 +0000 UTC m=+21.834112644" watchObservedRunningTime="2026-09-10 21:10:45.900959113 +0000 UTC m=+21.834625929"
Sep 10 21:10:46.672655 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:46.672624    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:46.672918 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:46.672741    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:46.799478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:46.799398    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rlv9n" event={"ID":"ff0b971f-0f3c-45dd-8234-c68a65df7eae","Type":"ContainerStarted","Data":"8f504452f0e2076b8d25abb039df6bfd86494611e1e7cf80366e3867de6d3024"}
Sep 10 21:10:46.803793 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:46.803715    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk" event={"ID":"f311012a-e71c-45af-b64d-83e2245f52ad","Type":"ContainerStarted","Data":"b6476711c84d8479643242e7c1e8eb7f50d3a7aaa748bc7c66565190982946fa"}
Sep 10 21:10:46.803918 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:46.803869    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk" event={"ID":"f311012a-e71c-45af-b64d-83e2245f52ad","Type":"ContainerStarted","Data":"d46b27bee35aedecc026140ec12a975f28d4dda53efcfc25968d412005cb4531"}
Sep 10 21:10:46.803985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:46.803951    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk" event={"ID":"f311012a-e71c-45af-b64d-83e2245f52ad","Type":"ContainerStarted","Data":"f1a1df06c81667e94bed9a423b9c5f21ee38b9b31d2506ee1bb5a6890e5373db"}
Sep 10 21:10:46.804039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:46.804022    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk" event={"ID":"f311012a-e71c-45af-b64d-83e2245f52ad","Type":"ContainerStarted","Data":"e2b276012f5e2f31e0b6031ac24292b3829c8ba4708576c3a29e0a8eb13fce75"}
Sep 10 21:10:46.818303 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:46.818250    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rlv9n" podStartSLOduration=4.053113697 podStartE2EDuration="21.818230585s" totalImagesPullingTime="17.765116888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:46.818106658 +0000 UTC m=+22.751773477" watchObservedRunningTime="2026-09-10 21:10:46.818230585 +0000 UTC m=+22.751897401"
Sep 10 21:10:46.932962 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:46.932816    2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 21:10:47.640357 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:47.640215    2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T21:10:46.932835477Z","UUID":"7aa99d2c-6139-4da0-bf8a-4f8e33010cb3","Handler":null,"Name":"","Endpoint":""}
Sep 10 21:10:47.641923 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:47.641899    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 21:10:47.642040 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:47.641941    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 21:10:47.673419 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:47.673389    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:47.673419 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:47.673419    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:47.673637 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:47.673534    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:47.673637 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:47.673622    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:47.807767 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:47.807732    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf" event={"ID":"3fbc2efe-e578-40d2-8897-ddcb48ef5588","Type":"ContainerStarted","Data":"2bec80d929debc4aab11576cb52435cb26f58660345b5b619ed41d5fdf10dfd5"}
Sep 10 21:10:48.672900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:48.672861    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:48.673130 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:48.672994    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:48.813054 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:48.813011    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf" event={"ID":"3fbc2efe-e578-40d2-8897-ddcb48ef5588","Type":"ContainerStarted","Data":"c9f16ddce7494865cc4d69180176113081fa2f5737a0de89ba472b407807cdc8"}
Sep 10 21:10:48.818151 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:48.818118    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk" event={"ID":"f311012a-e71c-45af-b64d-83e2245f52ad","Type":"ContainerStarted","Data":"47f29191c40438051dda8b2abb6b3538379c2ca0475f49c637fc5e6d62394c43"}
Sep 10 21:10:48.837224 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:48.837170    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48lxf" podStartSLOduration=4.936795909 podStartE2EDuration="24.837152186s" totalImagesPullingTime="19.900356277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:48.836700949 +0000 UTC m=+24.770367765" watchObservedRunningTime="2026-09-10 21:10:48.837152186 +0000 UTC m=+24.770819001"
Sep 10 21:10:49.120147 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:49.120109    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:49.120755 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:49.120736    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:49.673073 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:49.673030    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:49.673300 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:49.673030    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:49.673300 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:49.673149    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:49.673300 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:49.673221    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:50.673024 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:50.672990    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:50.673469 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:50.673111    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:51.673329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:51.673091    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:51.673329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:51.673091    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:51.673816 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:51.673355    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:51.673816 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:51.673422    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:51.826364 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:51.826333    2573 generic.go:356] "Generic (PLEG): container finished" podID="fa938a91-1541-485a-b88f-151697650d2a" containerID="40a3ec08a5d2090669e23995d0741f629e1500dde0a7ae28e420a792d906211e" exitCode=0
Sep 10 21:10:51.826524 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:51.826419    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fc4cj" event={"ID":"fa938a91-1541-485a-b88f-151697650d2a","Type":"ContainerDied","Data":"40a3ec08a5d2090669e23995d0741f629e1500dde0a7ae28e420a792d906211e"}
Sep 10 21:10:51.830733 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:51.830700    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk" event={"ID":"f311012a-e71c-45af-b64d-83e2245f52ad","Type":"ContainerStarted","Data":"623e1e5577da8501eee5dffc279911b77a134022f51e0c2bf0c8c898b7049eda"}
Sep 10 21:10:51.831012 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:51.830996    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:51.847135 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:51.847111    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:51.875349 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:51.875296    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk" podStartSLOduration=10.102450369 podStartE2EDuration="27.875276745s" totalImagesPullingTime="17.772826376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:51.874827355 +0000 UTC m=+27.808494167" watchObservedRunningTime="2026-09-10 21:10:51.875276745 +0000 UTC m=+27.808943560"
Sep 10 21:10:52.673108 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:52.673072    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:52.673296 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:52.673254    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:52.835868 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:52.835839    2573 generic.go:356] "Generic (PLEG): container finished" podID="fa938a91-1541-485a-b88f-151697650d2a" containerID="2bdf86b61fd053bbc895b95eeff26f8fbdc10db13d651077e1fe4e69a5f99417" exitCode=0
Sep 10 21:10:52.836512 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:52.835926    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fc4cj" event={"ID":"fa938a91-1541-485a-b88f-151697650d2a","Type":"ContainerDied","Data":"2bdf86b61fd053bbc895b95eeff26f8fbdc10db13d651077e1fe4e69a5f99417"}
Sep 10 21:10:52.836512 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:52.836422    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:52.836512 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:52.836491    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:52.851830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:52.851800    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:10:53.125656 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.125622    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z8k59"]
Sep 10 21:10:53.125815 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.125769    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:53.125919 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:53.125901    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:53.128811 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.128771    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vqprq"]
Sep 10 21:10:53.128943 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.128907    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:53.129041 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:53.129023    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:53.129690 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.129667    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4zmfv"]
Sep 10 21:10:53.129795 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.129769    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:53.129897 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:53.129877    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:53.766824 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.766619    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:53.767004 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.766900    2573 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 21:10:53.767273 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.767251    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vttxv"
Sep 10 21:10:53.840830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.840798    2573 generic.go:356] "Generic (PLEG): container finished" podID="fa938a91-1541-485a-b88f-151697650d2a" containerID="4b060923de7ae9d101bf347431d50db2e8a877ec0c6196b7350586e01bfdd941" exitCode=0
Sep 10 21:10:53.841354 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:53.840880    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fc4cj" event={"ID":"fa938a91-1541-485a-b88f-151697650d2a","Type":"ContainerDied","Data":"4b060923de7ae9d101bf347431d50db2e8a877ec0c6196b7350586e01bfdd941"}
Sep 10 21:10:54.673948 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:54.673887    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:54.674158 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:54.673999    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:54.674158 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:54.674041    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:54.674158 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:54.674058    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:54.674158 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:54.674114    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:54.674343 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:54.674190    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:56.672643 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:56.672608    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:56.673188 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:56.672608    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:56.673188 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:56.672738    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:56.673188 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:56.672831    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:56.673188 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:56.672613    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:56.673188 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:56.672911    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:58.186604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:58.186558    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:58.186604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:58.186621    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:58.187164 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.186724    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:58.187164 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.186735    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:58.187164 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.186810    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret podName:1f372f51-ce6b-46c2-9438-df2f95851903 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.186795331 +0000 UTC m=+66.120462143 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret") pod "global-pull-secret-syncer-z8k59" (UID: "1f372f51-ce6b-46c2-9438-df2f95851903") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:58.187164 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.186827    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs podName:87136fc5-90a6-4728-9410-a51d6050e497 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.186820328 +0000 UTC m=+66.120487121 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs") pod "network-metrics-daemon-vqprq" (UID: "87136fc5-90a6-4728-9410-a51d6050e497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:58.287674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:58.287625    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wzp\" (UniqueName: \"kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp\") pod \"network-check-target-4zmfv\" (UID: \"492738f6-5f08-4101-837f-87cf59c8e113\") " pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:58.287880 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.287818    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:58.287880 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.287841    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:58.287880 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.287850    2573 projected.go:196] Error preparing data for projected volume kube-api-access-h4wzp for pod openshift-network-diagnostics/network-check-target-4zmfv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:58.288046 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.287907    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp podName:492738f6-5f08-4101-837f-87cf59c8e113 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.287887614 +0000 UTC m=+66.221554413 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4wzp" (UniqueName: "kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp") pod "network-check-target-4zmfv" (UID: "492738f6-5f08-4101-837f-87cf59c8e113") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:58.672887 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:58.672847    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:10:58.673085 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:58.672847    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:10:58.673085 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:58.672981    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:10:58.673085 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.672984    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqprq" podUID="87136fc5-90a6-4728-9410-a51d6050e497"
Sep 10 21:10:58.673085 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.673066    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4zmfv" podUID="492738f6-5f08-4101-837f-87cf59c8e113"
Sep 10 21:10:58.673281 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:58.673129    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8k59" podUID="1f372f51-ce6b-46c2-9438-df2f95851903"
Sep 10 21:10:59.357210 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.357183    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-74.ec2.internal" event="NodeReady"
Sep 10 21:10:59.357595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.357291    2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 21:10:59.402566 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.402527    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dqff7"]
Sep 10 21:10:59.427052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.427015    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4mnm5"]
Sep 10 21:10:59.427210 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.427144    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.429393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.429370    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 21:10:59.429393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.429378    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 21:10:59.429561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.429497    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jpcjb\""
Sep 10 21:10:59.451146 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.451118    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dqff7"]
Sep 10 21:10:59.451291 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.451156    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4mnm5"]
Sep 10 21:10:59.451549 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.451521    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4mnm5"
Sep 10 21:10:59.454748 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.454721    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 21:10:59.454902 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.454721    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2h9gs\""
Sep 10 21:10:59.454902 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.454825    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 21:10:59.454902 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.454730    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 21:10:59.499552 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.499506    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.499751 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.499566    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-config-volume\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.499751 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.499631    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-tmp-dir\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.499751 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.499652    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7vlt\" (UniqueName: \"kubernetes.io/projected/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-kube-api-access-n7vlt\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.600864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.600825    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp6ws\" (UniqueName: \"kubernetes.io/projected/f701872b-3d4b-4030-9399-56e4fdc01097-kube-api-access-vp6ws\") pod \"ingress-canary-4mnm5\" (UID: \"f701872b-3d4b-4030-9399-56e4fdc01097\") " pod="openshift-ingress-canary/ingress-canary-4mnm5"
Sep 10 21:10:59.601034 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.600873    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-tmp-dir\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.601034 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.600898    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7vlt\" (UniqueName: \"kubernetes.io/projected/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-kube-api-access-n7vlt\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.601034 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.600950    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.601034 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.600981    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f701872b-3d4b-4030-9399-56e4fdc01097-cert\") pod \"ingress-canary-4mnm5\" (UID: \"f701872b-3d4b-4030-9399-56e4fdc01097\") " pod="openshift-ingress-canary/ingress-canary-4mnm5"
Sep 10 21:10:59.601034 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.601013    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-config-volume\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.601264 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:59.601108    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:59.601264 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:10:59.601169    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls podName:d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:00.10115067 +0000 UTC m=+36.034817467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls") pod "dns-default-dqff7" (UID: "d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:59.601381 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.601360    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-tmp-dir\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.601599 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.601583    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-config-volume\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.611236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.611058    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7vlt\" (UniqueName: \"kubernetes.io/projected/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-kube-api-access-n7vlt\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:10:59.702171 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.702131    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp6ws\" (UniqueName: \"kubernetes.io/projected/f701872b-3d4b-4030-9399-56e4fdc01097-kube-api-access-vp6ws\") pod \"ingress-canary-4mnm5\" (UID: \"f701872b-3d4b-4030-9399-56e4fdc01097\") " pod="openshift-ingress-canary/ingress-canary-4mnm5"
Sep 10 21:10:59.702333 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.702200    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f701872b-3d4b-4030-9399-56e4fdc01097-cert\") pod \"ingress-canary-4mnm5\" (UID: \"f701872b-3d4b-4030-9399-56e4fdc01097\") " pod="openshift-ingress-canary/ingress-canary-4mnm5"
Sep 10 21:10:59.704713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.704694    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f701872b-3d4b-4030-9399-56e4fdc01097-cert\") pod \"ingress-canary-4mnm5\" (UID: \"f701872b-3d4b-4030-9399-56e4fdc01097\") " pod="openshift-ingress-canary/ingress-canary-4mnm5"
Sep 10 21:10:59.710180 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.710157    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp6ws\" (UniqueName: \"kubernetes.io/projected/f701872b-3d4b-4030-9399-56e4fdc01097-kube-api-access-vp6ws\") pod \"ingress-canary-4mnm5\" (UID: \"f701872b-3d4b-4030-9399-56e4fdc01097\") " pod="openshift-ingress-canary/ingress-canary-4mnm5"
Sep 10 21:10:59.761206 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.761170    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4mnm5"
Sep 10 21:10:59.858386 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.858352    2573 generic.go:356] "Generic (PLEG): container finished" podID="fa938a91-1541-485a-b88f-151697650d2a" containerID="c688200cf1b0bc01f4d871cc6bace3afcbb5b6b516e172ee766e0101a8bfd4b7" exitCode=0
Sep 10 21:10:59.858571 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:10:59.858414    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fc4cj" event={"ID":"fa938a91-1541-485a-b88f-151697650d2a","Type":"ContainerDied","Data":"c688200cf1b0bc01f4d871cc6bace3afcbb5b6b516e172ee766e0101a8bfd4b7"}
Sep 10 21:11:00.085332 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.085297    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4mnm5"]
Sep 10 21:11:00.089485 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:11:00.089440    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf701872b_3d4b_4030_9399_56e4fdc01097.slice/crio-78b7a066b1548ce35356bbc0c4edbcd09d23a4a2f09456acc6a1416b4cec1d76 WatchSource:0}: Error finding container 78b7a066b1548ce35356bbc0c4edbcd09d23a4a2f09456acc6a1416b4cec1d76: Status 404 returned error can't find the container with id 78b7a066b1548ce35356bbc0c4edbcd09d23a4a2f09456acc6a1416b4cec1d76
Sep 10 21:11:00.105017 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.104995    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:11:00.105161 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:00.105142    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:11:00.105221 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:00.105211    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls podName:d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:01.105191663 +0000 UTC m=+37.038858463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls") pod "dns-default-dqff7" (UID: "d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8") : secret "dns-default-metrics-tls" not found
Sep 10 21:11:00.673012 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.672973    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:11:00.673889 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.672973    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:11:00.673889 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.672973    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:11:00.676873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.676643    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 21:11:00.676873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.676715    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 21:11:00.676873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.676737    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2xxbx\""
Sep 10 21:11:00.676873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.676643    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xghkt\""
Sep 10 21:11:00.676873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.676801    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 21:11:00.676873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.676801    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 21:11:00.865429 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.865395    2573 generic.go:356] "Generic (PLEG): container finished" podID="fa938a91-1541-485a-b88f-151697650d2a" containerID="e3cacdc061c36537d68a55ed6fd3dfb269e1beace76cfbc2b76620bc4bed59d3" exitCode=0
Sep 10 21:11:00.865620 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.865510    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fc4cj" event={"ID":"fa938a91-1541-485a-b88f-151697650d2a","Type":"ContainerDied","Data":"e3cacdc061c36537d68a55ed6fd3dfb269e1beace76cfbc2b76620bc4bed59d3"}
Sep 10 21:11:00.867172 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:00.867131    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4mnm5" event={"ID":"f701872b-3d4b-4030-9399-56e4fdc01097","Type":"ContainerStarted","Data":"78b7a066b1548ce35356bbc0c4edbcd09d23a4a2f09456acc6a1416b4cec1d76"}
Sep 10 21:11:01.113934 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:01.113895    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:11:01.114104 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:01.114007    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:11:01.114104 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:01.114060    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls podName:d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:03.114046031 +0000 UTC m=+39.047712828 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls") pod "dns-default-dqff7" (UID: "d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8") : secret "dns-default-metrics-tls" not found
Sep 10 21:11:01.877666 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:01.877417    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fc4cj" event={"ID":"fa938a91-1541-485a-b88f-151697650d2a","Type":"ContainerStarted","Data":"0ad8f75d1650b754cff3a3aa09430957018f921c1aa044e8f95edfe114846e83"}
Sep 10 21:11:01.903087 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:01.903025    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fc4cj" podStartSLOduration=7.926431744 podStartE2EDuration="37.90300367s" totalImagesPullingTime="29.976571926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:01.902069338 +0000 UTC m=+37.835736153" watchObservedRunningTime="2026-09-10 21:11:01.90300367 +0000 UTC m=+37.836670488"
Sep 10 21:11:03.132045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:03.132004    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:11:03.132456 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:03.132135    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:11:03.132456 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:03.132190    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls podName:d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:07.132176019 +0000 UTC m=+43.065842812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls") pod "dns-default-dqff7" (UID: "d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8") : secret "dns-default-metrics-tls" not found
Sep 10 21:11:03.884080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:03.884043    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4mnm5" event={"ID":"f701872b-3d4b-4030-9399-56e4fdc01097","Type":"ContainerStarted","Data":"4ea42d2d77834dfeabe72a20cf69f88fbf36f80eba452dac7b27d812f55307ea"}
Sep 10 21:11:03.900162 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:03.900000    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4mnm5" podStartSLOduration=2.170591857 podStartE2EDuration="4.899982756s" totalImagesPullingTime="2.729390899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:03.899619044 +0000 UTC m=+39.833285861" watchObservedRunningTime="2026-09-10 21:11:03.899982756 +0000 UTC m=+39.833649571"
Sep 10 21:11:07.161672 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:07.161626    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:11:07.162147 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:07.161811    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:11:07.162147 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:07.161895    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls podName:d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:15.161875015 +0000 UTC m=+51.095541808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls") pod "dns-default-dqff7" (UID: "d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8") : secret "dns-default-metrics-tls" not found
Sep 10 21:11:15.215297 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:15.215253    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:11:15.215802 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:15.215427    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:11:15.215802 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:15.215495    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls podName:d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:31.215476883 +0000 UTC m=+67.149143677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls") pod "dns-default-dqff7" (UID: "d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8") : secret "dns-default-metrics-tls" not found
Sep 10 21:11:22.211597 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.211563    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-4d785"]
Sep 10 21:11:22.241542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.241510    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-4d785"]
Sep 10 21:11:22.241698 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.241633    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.244345 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.244322    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 21:11:22.244345 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.244339    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:11:22.244552 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.244355    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 21:11:22.244552 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.244340    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ksdfc\""
Sep 10 21:11:22.244552 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.244346    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 21:11:22.249001 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.248980    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 21:11:22.317456 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.317424    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"]
Sep 10 21:11:22.339459 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.339426    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"]
Sep 10 21:11:22.339617 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.339548    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:22.342588 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.342557    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 21:11:22.343317 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.343291    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 21:11:22.343576 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.343557    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 21:11:22.343846 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.343827    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 21:11:22.344214 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.344190    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mgqkw\""
Sep 10 21:11:22.353873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.353849    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"]
Sep 10 21:11:22.353987 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.353885    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"]
Sep 10 21:11:22.353987 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.353956    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:22.356147 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.356131    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 10 21:11:22.356231 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.356176    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:11:22.356329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.356308    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4pqfv\""
Sep 10 21:11:22.356434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.356343    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 10 21:11:22.363515 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.363490    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bef584d6-f102-4e09-998f-2622be652abc-trusted-ca\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.363645 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.363528    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bef584d6-f102-4e09-998f-2622be652abc-serving-cert\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.363645 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.363562    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bef584d6-f102-4e09-998f-2622be652abc-config\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.363645 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.363580    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ttnm\" (UniqueName: \"kubernetes.io/projected/bef584d6-f102-4e09-998f-2622be652abc-kube-api-access-2ttnm\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.417196 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.417162    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68575978f-cqs5p"]
Sep 10 21:11:22.439083 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.439051    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68575978f-cqs5p"]
Sep 10 21:11:22.439239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.439146    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.442799 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.442767    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 21:11:22.443111 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.443091    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pjnbl\""
Sep 10 21:11:22.443397 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.443384    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 21:11:22.443445 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.443400    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 21:11:22.448559 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.448535    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 21:11:22.464948 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.464876    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ttnm\" (UniqueName: \"kubernetes.io/projected/bef584d6-f102-4e09-998f-2622be652abc-kube-api-access-2ttnm\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.464948 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.464923    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ce51c5d4-3d30-415d-8b3c-e009a99848ec-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:22.465124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.464972    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:22.465124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.464994    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bef584d6-f102-4e09-998f-2622be652abc-trusted-ca\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.465124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.465030    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bef584d6-f102-4e09-998f-2622be652abc-serving-cert\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.465298 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.465180    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6522\" (UniqueName: \"kubernetes.io/projected/e42e8df8-5a5c-4a9f-9ee0-da5941025760-kube-api-access-z6522\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:22.465298 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.465220    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:22.465298 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.465256    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxlqr\" (UniqueName: \"kubernetes.io/projected/ce51c5d4-3d30-415d-8b3c-e009a99848ec-kube-api-access-qxlqr\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:22.465298 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.465293    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bef584d6-f102-4e09-998f-2622be652abc-config\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.465846 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.465828    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bef584d6-f102-4e09-998f-2622be652abc-config\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.465992 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.465975    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bef584d6-f102-4e09-998f-2622be652abc-trusted-ca\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.468668 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.468641    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bef584d6-f102-4e09-998f-2622be652abc-serving-cert\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.474685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.474662    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ttnm\" (UniqueName: \"kubernetes.io/projected/bef584d6-f102-4e09-998f-2622be652abc-kube-api-access-2ttnm\") pod \"console-operator-696884fb9c-4d785\" (UID: \"bef584d6-f102-4e09-998f-2622be652abc\") " pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.550871 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.550827    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:22.566106 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566073    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ce51c5d4-3d30-415d-8b3c-e009a99848ec-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:22.566220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566138    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:22.566220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566188    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-trusted-ca\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.566220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566205    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.566386 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566221    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h76fp\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-kube-api-access-h76fp\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.566386 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566240    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-ca-trust-extracted\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.566386 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:22.566308    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 21:11:22.566386 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566337    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-image-registry-private-configuration\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.566554 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:22.566391    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls podName:e42e8df8-5a5c-4a9f-9ee0-da5941025760 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:23.066369591 +0000 UTC m=+59.000036403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-wcjl2" (UID: "e42e8df8-5a5c-4a9f-9ee0-da5941025760") : secret "samples-operator-tls" not found
Sep 10 21:11:22.566554 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566455    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6522\" (UniqueName: \"kubernetes.io/projected/e42e8df8-5a5c-4a9f-9ee0-da5941025760-kube-api-access-z6522\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:22.566554 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566473    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:22.566554 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566493    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-installation-pull-secrets\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.566554 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566511    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxlqr\" (UniqueName: \"kubernetes.io/projected/ce51c5d4-3d30-415d-8b3c-e009a99848ec-kube-api-access-qxlqr\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:22.566554 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566534    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-certificates\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.566813 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:22.566592    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:22.566813 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566639    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-bound-sa-token\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.566813 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:22.566686    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls podName:ce51c5d4-3d30-415d-8b3c-e009a99848ec nodeName:}" failed. No retries permitted until 2026-09-10 21:11:23.066664104 +0000 UTC m=+59.000330897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4nfrp" (UID: "ce51c5d4-3d30-415d-8b3c-e009a99848ec") : secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:22.566913 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.566848    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ce51c5d4-3d30-415d-8b3c-e009a99848ec-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:22.577649 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.577622    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6522\" (UniqueName: \"kubernetes.io/projected/e42e8df8-5a5c-4a9f-9ee0-da5941025760-kube-api-access-z6522\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:22.577789 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.577624    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxlqr\" (UniqueName: \"kubernetes.io/projected/ce51c5d4-3d30-415d-8b3c-e009a99848ec-kube-api-access-qxlqr\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:22.667628 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.667588    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-installation-pull-secrets\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.667628 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.667634    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-certificates\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.667866 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.667659    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-bound-sa-token\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.667866 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.667744    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-trusted-ca\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.667978 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.667949    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.668091 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.667997    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h76fp\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-kube-api-access-h76fp\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.668091 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.668033    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-ca-trust-extracted\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.668091 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.668064    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-image-registry-private-configuration\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.668309 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:22.668065    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:11:22.668309 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:22.668173    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68575978f-cqs5p: secret "image-registry-tls" not found
Sep 10 21:11:22.668309 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:22.668234    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls podName:eebd3773-65d7-4a74-bc3a-00fc3d8f1fba nodeName:}" failed. No retries permitted until 2026-09-10 21:11:23.168213475 +0000 UTC m=+59.101880282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls") pod "image-registry-68575978f-cqs5p" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba") : secret "image-registry-tls" not found
Sep 10 21:11:22.668459 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.668314    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-certificates\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.668459 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.668379    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-ca-trust-extracted\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.669211 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.669189    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-trusted-ca\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.670358 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.670339    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-image-registry-private-configuration\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.670431 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.670381    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-installation-pull-secrets\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.676787 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.676744    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-bound-sa-token\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.676930 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.676913    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h76fp\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-kube-api-access-h76fp\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:22.835907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.835869    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-4d785"]
Sep 10 21:11:22.838563 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:11:22.838524    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbef584d6_f102_4e09_998f_2622be652abc.slice/crio-87655a7502dc76327675ace74d85743c45f353b4eec9614c8015248fbcbb7c3c WatchSource:0}: Error finding container 87655a7502dc76327675ace74d85743c45f353b4eec9614c8015248fbcbb7c3c: Status 404 returned error can't find the container with id 87655a7502dc76327675ace74d85743c45f353b4eec9614c8015248fbcbb7c3c
Sep 10 21:11:22.932348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:22.932307    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-4d785" event={"ID":"bef584d6-f102-4e09-998f-2622be652abc","Type":"ContainerStarted","Data":"87655a7502dc76327675ace74d85743c45f353b4eec9614c8015248fbcbb7c3c"}
Sep 10 21:11:23.072040 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:23.072000    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:23.072208 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:23.072075    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:23.072208 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:23.072152    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 21:11:23.072208 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:23.072169    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:23.072313 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:23.072220    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls podName:e42e8df8-5a5c-4a9f-9ee0-da5941025760 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:24.072205148 +0000 UTC m=+60.005871941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-wcjl2" (UID: "e42e8df8-5a5c-4a9f-9ee0-da5941025760") : secret "samples-operator-tls" not found
Sep 10 21:11:23.072313 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:23.072235    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls podName:ce51c5d4-3d30-415d-8b3c-e009a99848ec nodeName:}" failed. No retries permitted until 2026-09-10 21:11:24.072228961 +0000 UTC m=+60.005895753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4nfrp" (UID: "ce51c5d4-3d30-415d-8b3c-e009a99848ec") : secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:23.172641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:23.172556    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:23.172801 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:23.172702    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:11:23.172801 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:23.172718    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68575978f-cqs5p: secret "image-registry-tls" not found
Sep 10 21:11:23.172801 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:23.172797    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls podName:eebd3773-65d7-4a74-bc3a-00fc3d8f1fba nodeName:}" failed. No retries permitted until 2026-09-10 21:11:24.172760689 +0000 UTC m=+60.106427499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls") pod "image-registry-68575978f-cqs5p" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba") : secret "image-registry-tls" not found
Sep 10 21:11:24.080038 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:24.079997    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:24.080415 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:24.080075    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:24.080415 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:24.080150    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 21:11:24.080415 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:24.080164    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:24.080415 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:24.080214    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls podName:e42e8df8-5a5c-4a9f-9ee0-da5941025760 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:26.080197852 +0000 UTC m=+62.013864644 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-wcjl2" (UID: "e42e8df8-5a5c-4a9f-9ee0-da5941025760") : secret "samples-operator-tls" not found
Sep 10 21:11:24.080415 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:24.080231    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls podName:ce51c5d4-3d30-415d-8b3c-e009a99848ec nodeName:}" failed. No retries permitted until 2026-09-10 21:11:26.080222216 +0000 UTC m=+62.013889008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4nfrp" (UID: "ce51c5d4-3d30-415d-8b3c-e009a99848ec") : secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:24.181178 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:24.181134    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:24.181339 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:24.181239    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:11:24.181339 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:24.181260    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68575978f-cqs5p: secret "image-registry-tls" not found
Sep 10 21:11:24.181339 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:24.181313    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls podName:eebd3773-65d7-4a74-bc3a-00fc3d8f1fba nodeName:}" failed. No retries permitted until 2026-09-10 21:11:26.181298914 +0000 UTC m=+62.114965711 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls") pod "image-registry-68575978f-cqs5p" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba") : secret "image-registry-tls" not found
Sep 10 21:11:24.854117 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:24.854086    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bp6vk"
Sep 10 21:11:24.939080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:24.939006    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/0.log"
Sep 10 21:11:24.939080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:24.939048    2573 generic.go:356] "Generic (PLEG): container finished" podID="bef584d6-f102-4e09-998f-2622be652abc" containerID="e6df648688f7732277c284d7149e189f0358ccc1508208a6c62b3ae7ee6df49f" exitCode=255
Sep 10 21:11:24.939256 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:24.939092    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-4d785" event={"ID":"bef584d6-f102-4e09-998f-2622be652abc","Type":"ContainerDied","Data":"e6df648688f7732277c284d7149e189f0358ccc1508208a6c62b3ae7ee6df49f"}
Sep 10 21:11:24.939373 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:24.939355    2573 scope.go:117] "RemoveContainer" containerID="e6df648688f7732277c284d7149e189f0358ccc1508208a6c62b3ae7ee6df49f"
Sep 10 21:11:25.943175 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:25.943146    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:11:25.943601 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:25.943562    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/0.log"
Sep 10 21:11:25.943643 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:25.943597    2573 generic.go:356] "Generic (PLEG): container finished" podID="bef584d6-f102-4e09-998f-2622be652abc" containerID="68b2e43ebb836178b39f08a0246b6180fff24e9bb93979d316a1e301ff5cb27c" exitCode=255
Sep 10 21:11:25.943677 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:25.943659    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-4d785" event={"ID":"bef584d6-f102-4e09-998f-2622be652abc","Type":"ContainerDied","Data":"68b2e43ebb836178b39f08a0246b6180fff24e9bb93979d316a1e301ff5cb27c"}
Sep 10 21:11:25.943792 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:25.943758    2573 scope.go:117] "RemoveContainer" containerID="e6df648688f7732277c284d7149e189f0358ccc1508208a6c62b3ae7ee6df49f"
Sep 10 21:11:25.943980 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:25.943963    2573 scope.go:117] "RemoveContainer" containerID="68b2e43ebb836178b39f08a0246b6180fff24e9bb93979d316a1e301ff5cb27c"
Sep 10 21:11:25.944152 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:25.944129    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-4d785_openshift-console-operator(bef584d6-f102-4e09-998f-2622be652abc)\"" pod="openshift-console-operator/console-operator-696884fb9c-4d785" podUID="bef584d6-f102-4e09-998f-2622be652abc"
Sep 10 21:11:26.094984 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:26.094926    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:26.095163 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:26.095038    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:26.095163 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:26.095091    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 21:11:26.095163 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:26.095124    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:26.095260 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:26.095165    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls podName:e42e8df8-5a5c-4a9f-9ee0-da5941025760 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.095144209 +0000 UTC m=+66.028811023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-wcjl2" (UID: "e42e8df8-5a5c-4a9f-9ee0-da5941025760") : secret "samples-operator-tls" not found
Sep 10 21:11:26.095260 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:26.095179    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls podName:ce51c5d4-3d30-415d-8b3c-e009a99848ec nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.095173465 +0000 UTC m=+66.028840258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4nfrp" (UID: "ce51c5d4-3d30-415d-8b3c-e009a99848ec") : secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:26.195487 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:26.195385    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:26.195646 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:26.195512    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:11:26.195646 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:26.195525    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68575978f-cqs5p: secret "image-registry-tls" not found
Sep 10 21:11:26.195646 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:26.195577    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls podName:eebd3773-65d7-4a74-bc3a-00fc3d8f1fba nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.195561274 +0000 UTC m=+66.129228090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls") pod "image-registry-68575978f-cqs5p" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba") : secret "image-registry-tls" not found
Sep 10 21:11:26.947740 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:26.947712    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:11:26.948124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:26.948037    2573 scope.go:117] "RemoveContainer" containerID="68b2e43ebb836178b39f08a0246b6180fff24e9bb93979d316a1e301ff5cb27c"
Sep 10 21:11:26.948247 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:26.948227    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-4d785_openshift-console-operator(bef584d6-f102-4e09-998f-2622be652abc)\"" pod="openshift-console-operator/console-operator-696884fb9c-4d785" podUID="bef584d6-f102-4e09-998f-2622be652abc"
Sep 10 21:11:27.118296 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.118263    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rfmjw"]
Sep 10 21:11:27.120954 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.120926    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.123759 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.123731    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6nt9k\""
Sep 10 21:11:27.123874 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.123848    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 21:11:27.124034 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.124021    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 21:11:27.124596 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.124575    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 21:11:27.124730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.124711    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 21:11:27.138704 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.138678    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rfmjw"]
Sep 10 21:11:27.201394 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.201302    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87209e5d-d1b5-4488-bdb0-55fed9ab391c-data-volume\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.201394 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.201338    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.201394 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.201368    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87209e5d-d1b5-4488-bdb0-55fed9ab391c-crio-socket\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.201394 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.201390    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stbbs\" (UniqueName: \"kubernetes.io/projected/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-api-access-stbbs\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.201638 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.201472    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.302733 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.302699    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stbbs\" (UniqueName: \"kubernetes.io/projected/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-api-access-stbbs\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.302834 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.302758    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.302885 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.302845    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87209e5d-d1b5-4488-bdb0-55fed9ab391c-data-volume\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.302885 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.302861    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.302977 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.302886    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87209e5d-d1b5-4488-bdb0-55fed9ab391c-crio-socket\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.302977 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:27.302962    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.303049 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.302962    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87209e5d-d1b5-4488-bdb0-55fed9ab391c-crio-socket\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.303049 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:27.302967    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:27.303049 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:27.303033    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:27.803018088 +0000 UTC m=+63.736684880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.303152 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:27.303064    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:27.803046979 +0000 UTC m=+63.736713789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:27.303189 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.303173    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87209e5d-d1b5-4488-bdb0-55fed9ab391c-data-volume\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.313248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.313228    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stbbs\" (UniqueName: \"kubernetes.io/projected/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-api-access-stbbs\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.808284 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.808238    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.808468 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:27.808369    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.808468 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:27.808414    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:27.808468 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:27.808433    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:28.808417604 +0000 UTC m=+64.742084397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.808629 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:27.808500    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:27.808629 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:27.808555    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:28.808544032 +0000 UTC m=+64.742210829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:28.816003 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:28.815964    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:28.816406 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:28.816064    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:28.816406 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:28.816143    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:28.816406 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:28.816149    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:28.816406 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:28.816197    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.816183451 +0000 UTC m=+66.749850244 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:28.816406 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:28.816210    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.81620431 +0000 UTC m=+66.749871103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:29.021527 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:29.021498    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dwlgl_cfc5e2be-0dac-4a19-9c82-f70065c9a9b4/dns-node-resolver/0.log"
Sep 10 21:11:30.125673 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.125632    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:30.126075 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.125803    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:30.126075 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.125806    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:30.126075 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.125895    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls podName:ce51c5d4-3d30-415d-8b3c-e009a99848ec nodeName:}" failed. No retries permitted until 2026-09-10 21:11:38.125881299 +0000 UTC m=+74.059548091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4nfrp" (UID: "ce51c5d4-3d30-415d-8b3c-e009a99848ec") : secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:30.126075 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.125927    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 21:11:30.126075 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.125981    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls podName:e42e8df8-5a5c-4a9f-9ee0-da5941025760 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:38.125968043 +0000 UTC m=+74.059634841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-wcjl2" (UID: "e42e8df8-5a5c-4a9f-9ee0-da5941025760") : secret "samples-operator-tls" not found
Sep 10 21:11:30.223482 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.223455    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-svw8v_0e6d31a2-19e9-4970-937a-e30522ffc0df/node-ca/0.log"
Sep 10 21:11:30.226736 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.226714    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:11:30.226805 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.226768    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:30.226879 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.226854    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:11:30.226959 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.226937    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:11:30.226959 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.226950    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68575978f-cqs5p: secret "image-registry-tls" not found
Sep 10 21:11:30.227062 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.227002    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls podName:eebd3773-65d7-4a74-bc3a-00fc3d8f1fba nodeName:}" failed. No retries permitted until 2026-09-10 21:11:38.226987092 +0000 UTC m=+74.160653906 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls") pod "image-registry-68575978f-cqs5p" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba") : secret "image-registry-tls" not found
Sep 10 21:11:30.228959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.228940    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 21:11:30.229041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.228968    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 21:11:30.237448 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.237427    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 21:11:30.237526 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.237501    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs podName:87136fc5-90a6-4728-9410-a51d6050e497 nodeName:}" failed. No retries permitted until 2026-09-10 21:12:34.237483586 +0000 UTC m=+130.171150379 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs") pod "network-metrics-daemon-vqprq" (UID: "87136fc5-90a6-4728-9410-a51d6050e497") : secret "metrics-daemon-secret" not found
Sep 10 21:11:30.239559 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.239537    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1f372f51-ce6b-46c2-9438-df2f95851903-original-pull-secret\") pod \"global-pull-secret-syncer-z8k59\" (UID: \"1f372f51-ce6b-46c2-9438-df2f95851903\") " pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:11:30.327626 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.327592    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wzp\" (UniqueName: \"kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp\") pod \"network-check-target-4zmfv\" (UID: \"492738f6-5f08-4101-837f-87cf59c8e113\") " pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:11:30.330218 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.330190    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 21:11:30.339985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.339959    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 21:11:30.351160 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.351128    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4wzp\" (UniqueName: \"kubernetes.io/projected/492738f6-5f08-4101-837f-87cf59c8e113-kube-api-access-h4wzp\") pod \"network-check-target-4zmfv\" (UID: \"492738f6-5f08-4101-837f-87cf59c8e113\") " pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:11:30.394304 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.394222    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2xxbx\""
Sep 10 21:11:30.397957 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.397937    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8k59"
Sep 10 21:11:30.402070 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.402054    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:11:30.621707 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.621684    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4mnm5_f701872b-3d4b-4030-9399-56e4fdc01097/serve-healthcheck-canary/0.log"
Sep 10 21:11:30.684895 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.684803    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4zmfv"]
Sep 10 21:11:30.687684 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:11:30.687651    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod492738f6_5f08_4101_837f_87cf59c8e113.slice/crio-0b46b78d43de2221ffd47bd3fae3f14d6b5c42e1c72f00e152893b7af57851ad WatchSource:0}: Error finding container 0b46b78d43de2221ffd47bd3fae3f14d6b5c42e1c72f00e152893b7af57851ad: Status 404 returned error can't find the container with id 0b46b78d43de2221ffd47bd3fae3f14d6b5c42e1c72f00e152893b7af57851ad
Sep 10 21:11:30.689282 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.689259    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z8k59"]
Sep 10 21:11:30.691959 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:11:30.691933    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f372f51_ce6b_46c2_9438_df2f95851903.slice/crio-b5fdaa1b1281c5756d9e62c38dc5138928b55e2cdd5ac3de94475ff528fb423f WatchSource:0}: Error finding container b5fdaa1b1281c5756d9e62c38dc5138928b55e2cdd5ac3de94475ff528fb423f: Status 404 returned error can't find the container with id b5fdaa1b1281c5756d9e62c38dc5138928b55e2cdd5ac3de94475ff528fb423f
Sep 10 21:11:30.832071 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.832036    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:30.832266 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.832111    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:30.832266 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.832159    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:30.832266 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.832226    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:34.832211198 +0000 UTC m=+70.765878008 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:30.832266 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.832239    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:30.832440 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:30.832299    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:34.832283299 +0000 UTC m=+70.765950093 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:30.959844 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.959734    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z8k59" event={"ID":"1f372f51-ce6b-46c2-9438-df2f95851903","Type":"ContainerStarted","Data":"b5fdaa1b1281c5756d9e62c38dc5138928b55e2cdd5ac3de94475ff528fb423f"}
Sep 10 21:11:30.960836 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:30.960812    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4zmfv" event={"ID":"492738f6-5f08-4101-837f-87cf59c8e113","Type":"ContainerStarted","Data":"0b46b78d43de2221ffd47bd3fae3f14d6b5c42e1c72f00e152893b7af57851ad"}
Sep 10 21:11:31.236060 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:31.235961    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:11:31.236449 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:31.236145    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:11:31.236449 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:31.236231    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls podName:d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8 nodeName:}" failed. No retries permitted until 2026-09-10 21:12:03.23620922 +0000 UTC m=+99.169876031 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls") pod "dns-default-dqff7" (UID: "d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8") : secret "dns-default-metrics-tls" not found
Sep 10 21:11:32.551511 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:32.551467    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:32.551954 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:32.551544    2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:32.551954 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:32.551943    2573 scope.go:117] "RemoveContainer" containerID="68b2e43ebb836178b39f08a0246b6180fff24e9bb93979d316a1e301ff5cb27c"
Sep 10 21:11:32.552128 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:32.552108    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-4d785_openshift-console-operator(bef584d6-f102-4e09-998f-2622be652abc)\"" pod="openshift-console-operator/console-operator-696884fb9c-4d785" podUID="bef584d6-f102-4e09-998f-2622be652abc"
Sep 10 21:11:33.972618 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:33.972522    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4zmfv" event={"ID":"492738f6-5f08-4101-837f-87cf59c8e113","Type":"ContainerStarted","Data":"40c14f7f5273ab599217cd03a2444de5e260b48c9c5ab33f146e1e535339a977"}
Sep 10 21:11:33.973039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:33.972643    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:11:33.993575 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:33.993517    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4zmfv" podStartSLOduration=65.965948786 podStartE2EDuration="1m8.993473133s" totalImagesPullingTime="3.027524347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:33.988903184 +0000 UTC m=+69.922570001" watchObservedRunningTime="2026-09-10 21:11:33.993473133 +0000 UTC m=+69.927139949"
Sep 10 21:11:34.867629 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:34.867588    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:34.867852 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:34.867665    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:34.867852 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:34.867741    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:34.867852 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:34.867808    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:34.867852 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:34.867832    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:42.867812645 +0000 UTC m=+78.801479659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:34.868058 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:34.867863    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:42.867846593 +0000 UTC m=+78.801513390 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:35.979351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:35.979316    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z8k59" event={"ID":"1f372f51-ce6b-46c2-9438-df2f95851903","Type":"ContainerStarted","Data":"3b4e90ec02e4a806360330e4e7f36f45eac92af6781e3959fb96c4e48b141ee0"}
Sep 10 21:11:35.995645 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:35.995587    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z8k59" podStartSLOduration=66.422426767 podStartE2EDuration="1m10.9955712s" totalImagesPullingTime="4.573144433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:35.995163712 +0000 UTC m=+71.928830528" watchObservedRunningTime="2026-09-10 21:11:35.9955712 +0000 UTC m=+71.929238015"
Sep 10 21:11:38.194605 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.194563    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:38.195007 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:38.194727    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:38.195007 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.194741    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:38.195007 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:38.194823    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls podName:ce51c5d4-3d30-415d-8b3c-e009a99848ec nodeName:}" failed. No retries permitted until 2026-09-10 21:11:54.194803616 +0000 UTC m=+90.128470410 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4nfrp" (UID: "ce51c5d4-3d30-415d-8b3c-e009a99848ec") : secret "cluster-monitoring-operator-tls" not found
Sep 10 21:11:38.197104 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.197075    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e42e8df8-5a5c-4a9f-9ee0-da5941025760-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wcjl2\" (UID: \"e42e8df8-5a5c-4a9f-9ee0-da5941025760\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:38.264157 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.264126    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4pqfv\""
Sep 10 21:11:38.272039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.272015    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"
Sep 10 21:11:38.296189 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.296157    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:38.298578 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.298546    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls\") pod \"image-registry-68575978f-cqs5p\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") " pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:38.353804 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.351768    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pjnbl\""
Sep 10 21:11:38.359160 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.359115    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:38.539333 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.539248    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2"]
Sep 10 21:11:38.636111 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.636075    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68575978f-cqs5p"]
Sep 10 21:11:38.639757 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:11:38.639732    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebd3773_65d7_4a74_bc3a_00fc3d8f1fba.slice/crio-9b01f41066c544d8ce34d3d7848b3926bf7c149ca7294e14e24be5e46cedf2b0 WatchSource:0}: Error finding container 9b01f41066c544d8ce34d3d7848b3926bf7c149ca7294e14e24be5e46cedf2b0: Status 404 returned error can't find the container with id 9b01f41066c544d8ce34d3d7848b3926bf7c149ca7294e14e24be5e46cedf2b0
Sep 10 21:11:38.989315 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.989272    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68575978f-cqs5p" event={"ID":"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba","Type":"ContainerStarted","Data":"257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333"}
Sep 10 21:11:38.989509 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.989340    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68575978f-cqs5p" event={"ID":"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba","Type":"ContainerStarted","Data":"9b01f41066c544d8ce34d3d7848b3926bf7c149ca7294e14e24be5e46cedf2b0"}
Sep 10 21:11:38.989509 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.989371    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:11:38.990469 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:38.990439    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2" event={"ID":"e42e8df8-5a5c-4a9f-9ee0-da5941025760","Type":"ContainerStarted","Data":"16768b230d36dc39e74ced2227cc940debb45b833bd919b7d9ced5a657690b8a"}
Sep 10 21:11:39.015132 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:39.015064    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68575978f-cqs5p" podStartSLOduration=17.015042236 podStartE2EDuration="17.015042236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:39.013834244 +0000 UTC m=+74.947501050" watchObservedRunningTime="2026-09-10 21:11:39.015042236 +0000 UTC m=+74.948709051"
Sep 10 21:11:40.998952 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:40.998919    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2" event={"ID":"e42e8df8-5a5c-4a9f-9ee0-da5941025760","Type":"ContainerStarted","Data":"996a1cf0aff3298433e0a36488d930b78bb8a9d23824f3b8fa799051f4ec27ae"}
Sep 10 21:11:40.999330 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:40.998971    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2" event={"ID":"e42e8df8-5a5c-4a9f-9ee0-da5941025760","Type":"ContainerStarted","Data":"b917508ce1b7fc97ba03f214912381bad457e73114fb8299658390d2e5fac464"}
Sep 10 21:11:41.014438 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:41.014379    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wcjl2" podStartSLOduration=17.306948447 podStartE2EDuration="19.014357748s" totalImagesPullingTime="1.707409301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:41.014028742 +0000 UTC m=+76.947695558" watchObservedRunningTime="2026-09-10 21:11:41.014357748 +0000 UTC m=+76.948024569"
Sep 10 21:11:42.936713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:42.936658    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:42.937091 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:42.936732    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:42.937091 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:42.936857    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:42.937091 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:11:42.936910    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls podName:87209e5d-d1b5-4488-bdb0-55fed9ab391c nodeName:}" failed. No retries permitted until 2026-09-10 21:11:58.9368978 +0000 UTC m=+94.870564592 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfmjw" (UID: "87209e5d-d1b5-4488-bdb0-55fed9ab391c") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:42.937224 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:42.937209    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87209e5d-d1b5-4488-bdb0-55fed9ab391c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:43.673326 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:43.673290    2573 scope.go:117] "RemoveContainer" containerID="68b2e43ebb836178b39f08a0246b6180fff24e9bb93979d316a1e301ff5cb27c"
Sep 10 21:11:44.010931 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:44.010853    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:11:44.010931 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:44.010909    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-4d785" event={"ID":"bef584d6-f102-4e09-998f-2622be652abc","Type":"ContainerStarted","Data":"2d2ef3f32860a8825b820f4273efb58e068bb1ccf61c9df3560817e119dc2912"}
Sep 10 21:11:44.011418 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:44.011175    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:44.027192 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:44.027140    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-4d785" podStartSLOduration=20.179497714 podStartE2EDuration="22.027126252s" totalImagesPullingTime="1.847628538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:44.026470019 +0000 UTC m=+79.960136834" watchObservedRunningTime="2026-09-10 21:11:44.027126252 +0000 UTC m=+79.960793065"
Sep 10 21:11:44.247944 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:44.247913    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-4d785"
Sep 10 21:11:53.346213 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.346099    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"]
Sep 10 21:11:53.348861 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.348844    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.351794 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.351751    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 21:11:53.352129 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.351866    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 21:11:53.352535 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.352517    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 21:11:53.358193 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.358177    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 21:11:53.364853 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.364830    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"]
Sep 10 21:11:53.394515 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.394479    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68575978f-cqs5p"]
Sep 10 21:11:53.399065 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.399034    2573 patch_prober.go:28] interesting pod/image-registry-68575978f-cqs5p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 21:11:53.399208 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.399110    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-68575978f-cqs5p" podUID="eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:11:53.419627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.419594    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/99c5fe4c-da08-4f00-847f-a1797c88fc12-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f78c88554-bs459\" (UID: \"99c5fe4c-da08-4f00-847f-a1797c88fc12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.419627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.419631    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j75pf\" (UniqueName: \"kubernetes.io/projected/99c5fe4c-da08-4f00-847f-a1797c88fc12-kube-api-access-j75pf\") pod \"klusterlet-addon-workmgr-7f78c88554-bs459\" (UID: \"99c5fe4c-da08-4f00-847f-a1797c88fc12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.419898 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.419682    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99c5fe4c-da08-4f00-847f-a1797c88fc12-tmp\") pod \"klusterlet-addon-workmgr-7f78c88554-bs459\" (UID: \"99c5fe4c-da08-4f00-847f-a1797c88fc12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.493486 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.493454    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c8bb7b487-s8pmp"]
Sep 10 21:11:53.495516 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.495500    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.510658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.510616    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c8bb7b487-s8pmp"]
Sep 10 21:11:53.520828 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.520798    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/99c5fe4c-da08-4f00-847f-a1797c88fc12-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f78c88554-bs459\" (UID: \"99c5fe4c-da08-4f00-847f-a1797c88fc12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.520975 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.520833    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j75pf\" (UniqueName: \"kubernetes.io/projected/99c5fe4c-da08-4f00-847f-a1797c88fc12-kube-api-access-j75pf\") pod \"klusterlet-addon-workmgr-7f78c88554-bs459\" (UID: \"99c5fe4c-da08-4f00-847f-a1797c88fc12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.520975 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.520893    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99c5fe4c-da08-4f00-847f-a1797c88fc12-tmp\") pod \"klusterlet-addon-workmgr-7f78c88554-bs459\" (UID: \"99c5fe4c-da08-4f00-847f-a1797c88fc12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.521337 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.521315    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99c5fe4c-da08-4f00-847f-a1797c88fc12-tmp\") pod \"klusterlet-addon-workmgr-7f78c88554-bs459\" (UID: \"99c5fe4c-da08-4f00-847f-a1797c88fc12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.523280 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.523254    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/99c5fe4c-da08-4f00-847f-a1797c88fc12-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f78c88554-bs459\" (UID: \"99c5fe4c-da08-4f00-847f-a1797c88fc12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.535484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.535449    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j75pf\" (UniqueName: \"kubernetes.io/projected/99c5fe4c-da08-4f00-847f-a1797c88fc12-kube-api-access-j75pf\") pod \"klusterlet-addon-workmgr-7f78c88554-bs459\" (UID: \"99c5fe4c-da08-4f00-847f-a1797c88fc12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.621975 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.621882    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a26f49-cd50-463a-ae10-38eb845bcbe7-bound-sa-token\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.621975 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.621925    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a26f49-cd50-463a-ae10-38eb845bcbe7-ca-trust-extracted\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.621975 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.621942    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a26f49-cd50-463a-ae10-38eb845bcbe7-registry-certificates\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.621975 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.621964    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a26f49-cd50-463a-ae10-38eb845bcbe7-image-registry-private-configuration\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.622216 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.622021    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a26f49-cd50-463a-ae10-38eb845bcbe7-trusted-ca\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.622216 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.622058    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a26f49-cd50-463a-ae10-38eb845bcbe7-registry-tls\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.622216 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.622121    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a26f49-cd50-463a-ae10-38eb845bcbe7-installation-pull-secrets\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.622216 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.622186    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg28r\" (UniqueName: \"kubernetes.io/projected/f8a26f49-cd50-463a-ae10-38eb845bcbe7-kube-api-access-fg28r\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.659392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.659351    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:53.723170 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.723138    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a26f49-cd50-463a-ae10-38eb845bcbe7-ca-trust-extracted\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.723338 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.723176    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a26f49-cd50-463a-ae10-38eb845bcbe7-registry-certificates\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.723395 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.723360    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a26f49-cd50-463a-ae10-38eb845bcbe7-image-registry-private-configuration\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.723455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.723426    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a26f49-cd50-463a-ae10-38eb845bcbe7-trusted-ca\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.723504 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.723491    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a26f49-cd50-463a-ae10-38eb845bcbe7-registry-tls\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.723588 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.723563    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a26f49-cd50-463a-ae10-38eb845bcbe7-ca-trust-extracted\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.723640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.723601    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a26f49-cd50-463a-ae10-38eb845bcbe7-installation-pull-secrets\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.723697 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.723670    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg28r\" (UniqueName: \"kubernetes.io/projected/f8a26f49-cd50-463a-ae10-38eb845bcbe7-kube-api-access-fg28r\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.723754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.723703    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a26f49-cd50-463a-ae10-38eb845bcbe7-bound-sa-token\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.724387 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.724367    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a26f49-cd50-463a-ae10-38eb845bcbe7-registry-certificates\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.724687 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.724664    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a26f49-cd50-463a-ae10-38eb845bcbe7-trusted-ca\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.726813 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.726724    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a26f49-cd50-463a-ae10-38eb845bcbe7-registry-tls\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.727142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.727123    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a26f49-cd50-463a-ae10-38eb845bcbe7-installation-pull-secrets\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.727733 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.727715    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a26f49-cd50-463a-ae10-38eb845bcbe7-image-registry-private-configuration\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.735348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.735318    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg28r\" (UniqueName: \"kubernetes.io/projected/f8a26f49-cd50-463a-ae10-38eb845bcbe7-kube-api-access-fg28r\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.735642 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.735623    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a26f49-cd50-463a-ae10-38eb845bcbe7-bound-sa-token\") pod \"image-registry-7c8bb7b487-s8pmp\" (UID: \"f8a26f49-cd50-463a-ae10-38eb845bcbe7\") " pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.804424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.804387    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:53.925954 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:53.925877    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"]
Sep 10 21:11:53.928911 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:11:53.928887    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c5fe4c_da08_4f00_847f_a1797c88fc12.slice/crio-628935f1497a0f3c1b739862adce5b0de2d1189403ee9efcad4b38a728966a8a WatchSource:0}: Error finding container 628935f1497a0f3c1b739862adce5b0de2d1189403ee9efcad4b38a728966a8a: Status 404 returned error can't find the container with id 628935f1497a0f3c1b739862adce5b0de2d1189403ee9efcad4b38a728966a8a
Sep 10 21:11:54.043653 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:54.043616    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459" event={"ID":"99c5fe4c-da08-4f00-847f-a1797c88fc12","Type":"ContainerStarted","Data":"628935f1497a0f3c1b739862adce5b0de2d1189403ee9efcad4b38a728966a8a"}
Sep 10 21:11:54.073589 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:54.073557    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c8bb7b487-s8pmp"]
Sep 10 21:11:54.076455 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:11:54.076426    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a26f49_cd50_463a_ae10_38eb845bcbe7.slice/crio-691d210bc788028d8cb8c86d9e88eb7a683d5cdf48eaf516a4e25eb6f3736348 WatchSource:0}: Error finding container 691d210bc788028d8cb8c86d9e88eb7a683d5cdf48eaf516a4e25eb6f3736348: Status 404 returned error can't find the container with id 691d210bc788028d8cb8c86d9e88eb7a683d5cdf48eaf516a4e25eb6f3736348
Sep 10 21:11:54.228556 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:54.228461    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:54.230830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:54.230805    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce51c5d4-3d30-415d-8b3c-e009a99848ec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4nfrp\" (UID: \"ce51c5d4-3d30-415d-8b3c-e009a99848ec\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:54.452363 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:54.452323    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mgqkw\""
Sep 10 21:11:54.461175 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:54.461129    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"
Sep 10 21:11:54.728394 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:54.728355    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp"]
Sep 10 21:11:54.730970 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:11:54.730949    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce51c5d4_3d30_415d_8b3c_e009a99848ec.slice/crio-24717649bd93c82717186f7e09a3d3d4c4fea88970644d4652d716a57fc2864f WatchSource:0}: Error finding container 24717649bd93c82717186f7e09a3d3d4c4fea88970644d4652d716a57fc2864f: Status 404 returned error can't find the container with id 24717649bd93c82717186f7e09a3d3d4c4fea88970644d4652d716a57fc2864f
Sep 10 21:11:55.049185 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:55.049138    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp" event={"ID":"ce51c5d4-3d30-415d-8b3c-e009a99848ec","Type":"ContainerStarted","Data":"24717649bd93c82717186f7e09a3d3d4c4fea88970644d4652d716a57fc2864f"}
Sep 10 21:11:55.051965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:55.051919    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp" event={"ID":"f8a26f49-cd50-463a-ae10-38eb845bcbe7","Type":"ContainerStarted","Data":"2669811d65edb6defd89f8d6e3ddfbb8b9e65e57ffc79f4344e88024fd7031e8"}
Sep 10 21:11:55.052120 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:55.051970    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp" event={"ID":"f8a26f49-cd50-463a-ae10-38eb845bcbe7","Type":"ContainerStarted","Data":"691d210bc788028d8cb8c86d9e88eb7a683d5cdf48eaf516a4e25eb6f3736348"}
Sep 10 21:11:55.052120 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:55.052045    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:11:55.081599 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:55.081532    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp" podStartSLOduration=2.08150785 podStartE2EDuration="2.08150785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:55.078981272 +0000 UTC m=+91.012648081" watchObservedRunningTime="2026-09-10 21:11:55.08150785 +0000 UTC m=+91.015174666"
Sep 10 21:11:58.968408 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:58.968365    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:58.970720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:58.970698    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87209e5d-d1b5-4488-bdb0-55fed9ab391c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfmjw\" (UID: \"87209e5d-d1b5-4488-bdb0-55fed9ab391c\") " pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:59.069747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:59.069711    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459" event={"ID":"99c5fe4c-da08-4f00-847f-a1797c88fc12","Type":"ContainerStarted","Data":"ad1b334c628051e1a48b6d60e245ebf7723cac03702e26bef10dc19cc7cfd773"}
Sep 10 21:11:59.069955 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:59.069899    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:59.072397 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:59.072354    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp" event={"ID":"ce51c5d4-3d30-415d-8b3c-e009a99848ec","Type":"ContainerStarted","Data":"988d03100e6d6fbc6ce4f0edc975ab9b227ab2c63dcac4fcdd83897004e4334a"}
Sep 10 21:11:59.075088 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:59.075063    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459"
Sep 10 21:11:59.088682 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:59.088630    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f78c88554-bs459" podStartSLOduration=1.94055807 podStartE2EDuration="6.088616963s" totalImagesPullingTime="4.148058893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:59.087046284 +0000 UTC m=+95.020713111" watchObservedRunningTime="2026-09-10 21:11:59.088616963 +0000 UTC m=+95.022283778"
Sep 10 21:11:59.102675 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:59.102627    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4nfrp" podStartSLOduration=33.773128499 podStartE2EDuration="37.102611623s" totalImagesPullingTime="3.329483124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:59.101729924 +0000 UTC m=+95.035396740" watchObservedRunningTime="2026-09-10 21:11:59.102611623 +0000 UTC m=+95.036278472"
Sep 10 21:11:59.230292 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:59.230192    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rfmjw"
Sep 10 21:11:59.503537 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:11:59.503459    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rfmjw"]
Sep 10 21:11:59.506425 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:11:59.506393    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87209e5d_d1b5_4488_bdb0_55fed9ab391c.slice/crio-efeb7971655d18dc117e1ccf35b482e34597799fb586cff51e47404ca09817ba WatchSource:0}: Error finding container efeb7971655d18dc117e1ccf35b482e34597799fb586cff51e47404ca09817ba: Status 404 returned error can't find the container with id efeb7971655d18dc117e1ccf35b482e34597799fb586cff51e47404ca09817ba
Sep 10 21:12:00.077613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:00.077577    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rfmjw" event={"ID":"87209e5d-d1b5-4488-bdb0-55fed9ab391c","Type":"ContainerStarted","Data":"67ab834a298b6213b3a49107b0eff6ec498e65c03b09b46a71fd69e14f243909"}
Sep 10 21:12:00.078133 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:00.077646    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rfmjw" event={"ID":"87209e5d-d1b5-4488-bdb0-55fed9ab391c","Type":"ContainerStarted","Data":"efeb7971655d18dc117e1ccf35b482e34597799fb586cff51e47404ca09817ba"}
Sep 10 21:12:01.084742 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.084702    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rfmjw" event={"ID":"87209e5d-d1b5-4488-bdb0-55fed9ab391c","Type":"ContainerStarted","Data":"a61c439089e8d0a3cfc963590db77392feb9b926c2d5db8b6e41ea068fd05888"}
Sep 10 21:12:01.654677 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.654645    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"]
Sep 10 21:12:01.659096 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.659067    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.662204 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.662182    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-sx2mj\""
Sep 10 21:12:01.662330 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.662184    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 21:12:01.662330 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.662221    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 21:12:01.662330 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.662186    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 21:12:01.668411 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.668384    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"]
Sep 10 21:12:01.789447 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.789411    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d301662-abdf-41e0-901a-4a171c47e84a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.789447 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.789455    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d301662-abdf-41e0-901a-4a171c47e84a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.789656 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.789522    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w52mc\" (UniqueName: \"kubernetes.io/projected/2d301662-abdf-41e0-901a-4a171c47e84a-kube-api-access-w52mc\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.789656 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.789539    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d301662-abdf-41e0-901a-4a171c47e84a-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.890834 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.890796    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w52mc\" (UniqueName: \"kubernetes.io/projected/2d301662-abdf-41e0-901a-4a171c47e84a-kube-api-access-w52mc\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.891036 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.890847    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d301662-abdf-41e0-901a-4a171c47e84a-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.891036 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.890893    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d301662-abdf-41e0-901a-4a171c47e84a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.891036 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.890941    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d301662-abdf-41e0-901a-4a171c47e84a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.891745 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.891715    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d301662-abdf-41e0-901a-4a171c47e84a-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.893373 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.893342    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d301662-abdf-41e0-901a-4a171c47e84a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.893373 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.893356    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d301662-abdf-41e0-901a-4a171c47e84a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.904295 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.904266    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w52mc\" (UniqueName: \"kubernetes.io/projected/2d301662-abdf-41e0-901a-4a171c47e84a-kube-api-access-w52mc\") pod \"prometheus-operator-68bcd7f987-25sjm\" (UID: \"2d301662-abdf-41e0-901a-4a171c47e84a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:01.968330 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:01.968228    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"
Sep 10 21:12:02.090656 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:02.090618    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rfmjw" event={"ID":"87209e5d-d1b5-4488-bdb0-55fed9ab391c","Type":"ContainerStarted","Data":"fde3ae3f2b4c7e5b7f8515ad6e580c48d1bdca101a7cb57aca05855e6e114213"}
Sep 10 21:12:02.114097 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:02.114045    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rfmjw" podStartSLOduration=33.015252421 podStartE2EDuration="35.11402853s" totalImagesPullingTime="2.098776109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:11:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:02.113541521 +0000 UTC m=+98.047208335" watchObservedRunningTime="2026-09-10 21:12:02.11402853 +0000 UTC m=+98.047695369"
Sep 10 21:12:02.252007 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:02.251969    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-25sjm"]
Sep 10 21:12:02.254362 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:12:02.254336    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d301662_abdf_41e0_901a_4a171c47e84a.slice/crio-586b39317235a6740985d32083ce3c92c1dfdb3f0f74b7ea3103955e8b095557 WatchSource:0}: Error finding container 586b39317235a6740985d32083ce3c92c1dfdb3f0f74b7ea3103955e8b095557: Status 404 returned error can't find the container with id 586b39317235a6740985d32083ce3c92c1dfdb3f0f74b7ea3103955e8b095557
Sep 10 21:12:03.096409 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:03.096369    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm" event={"ID":"2d301662-abdf-41e0-901a-4a171c47e84a","Type":"ContainerStarted","Data":"586b39317235a6740985d32083ce3c92c1dfdb3f0f74b7ea3103955e8b095557"}
Sep 10 21:12:03.304114 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:03.304072    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:12:03.306471 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:03.306439    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8-metrics-tls\") pod \"dns-default-dqff7\" (UID: \"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8\") " pod="openshift-dns/dns-default-dqff7"
Sep 10 21:12:03.338506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:03.338469    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jpcjb\""
Sep 10 21:12:03.347286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:03.347174    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dqff7"
Sep 10 21:12:03.400056 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:03.400017    2573 patch_prober.go:28] interesting pod/image-registry-68575978f-cqs5p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 21:12:03.400247 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:03.400087    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-68575978f-cqs5p" podUID="eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:12:03.921178 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:03.921096    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dqff7"]
Sep 10 21:12:03.925909 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:12:03.925865    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a4a217_d52e_4a0b_8adc_c8d0b4047dc8.slice/crio-6cca479ee14abe1dcc718ef46db507468c6f0ed198a8720a28d1d0611733ff69 WatchSource:0}: Error finding container 6cca479ee14abe1dcc718ef46db507468c6f0ed198a8720a28d1d0611733ff69: Status 404 returned error can't find the container with id 6cca479ee14abe1dcc718ef46db507468c6f0ed198a8720a28d1d0611733ff69
Sep 10 21:12:04.102707 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:04.102674    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dqff7" event={"ID":"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8","Type":"ContainerStarted","Data":"6cca479ee14abe1dcc718ef46db507468c6f0ed198a8720a28d1d0611733ff69"}
Sep 10 21:12:04.104447 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:04.104423    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm" event={"ID":"2d301662-abdf-41e0-901a-4a171c47e84a","Type":"ContainerStarted","Data":"a468cb4b648f3185f65d6c44ad117f7e2810b2c904a405dce2241921454f5bd5"}
Sep 10 21:12:04.104556 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:04.104467    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm" event={"ID":"2d301662-abdf-41e0-901a-4a171c47e84a","Type":"ContainerStarted","Data":"b0134dd0b5cc1c7760d9e1b8df65f02bcd4078b6061445bbbee9a4aaddb67814"}
Sep 10 21:12:04.121903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:04.121856    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-25sjm" podStartSLOduration=1.691695996 podStartE2EDuration="3.121841367s" totalImagesPullingTime="1.430145371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:04.121628035 +0000 UTC m=+100.055294849" watchObservedRunningTime="2026-09-10 21:12:04.121841367 +0000 UTC m=+100.055508181"
Sep 10 21:12:04.977713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:04.977676    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4zmfv"
Sep 10 21:12:06.028440 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.028345    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"]
Sep 10 21:12:06.030924 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.030901    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.033033 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.033009    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 21:12:06.033281 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.033259    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 21:12:06.033452 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.033428    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kvsf8\""
Sep 10 21:12:06.041735 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.041712    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"]
Sep 10 21:12:06.057278 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.057241    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rfvqt"]
Sep 10 21:12:06.059441 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.059423    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.061570 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.061549    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 21:12:06.061850 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.061833    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 21:12:06.061931 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.061894    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 21:12:06.061993 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.061894    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mjpkt\""
Sep 10 21:12:06.113047 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.113014    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dqff7" event={"ID":"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8","Type":"ContainerStarted","Data":"aab6126a2de15a951959129308afdc4c9263e2a4034167f26e8c3f12e4c01181"}
Sep 10 21:12:06.113220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.113067    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dqff7" event={"ID":"d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8","Type":"ContainerStarted","Data":"f2f31b40315047c4b81cf139454bbfef8c575305d80a3ddf1d3ec77770521d3f"}
Sep 10 21:12:06.113220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.113110    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dqff7"
Sep 10 21:12:06.128363 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.128327    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e637906-06fe-417c-8278-44962a5a80af-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.128363 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.128363    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e637906-06fe-417c-8278-44962a5a80af-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.128585 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.128396    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nzzb\" (UniqueName: \"kubernetes.io/projected/8e637906-06fe-417c-8278-44962a5a80af-kube-api-access-6nzzb\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.128585 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.128425    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e637906-06fe-417c-8278-44962a5a80af-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.133045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.132999    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dqff7" podStartSLOduration=65.427475292 podStartE2EDuration="1m7.132981858s" totalImagesPullingTime="1.705506566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:06.131979384 +0000 UTC m=+102.065646200" watchObservedRunningTime="2026-09-10 21:12:06.132981858 +0000 UTC m=+102.066648672"
Sep 10 21:12:06.229322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229279    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-wtmp\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.229322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229318    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25b0113e-d86c-464d-89d7-c79c0a4754a9-metrics-client-ca\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.229539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229351    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh2rf\" (UniqueName: \"kubernetes.io/projected/25b0113e-d86c-464d-89d7-c79c0a4754a9-kube-api-access-qh2rf\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.229539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229395    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.229539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229455    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e637906-06fe-417c-8278-44962a5a80af-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.229539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229481    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-tls\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.229539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229522    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-accelerators-collector-config\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.229733 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229569    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25b0113e-d86c-464d-89d7-c79c0a4754a9-sys\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.229733 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229590    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-textfile\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.229850 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229741    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e637906-06fe-417c-8278-44962a5a80af-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.229850 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229799    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e637906-06fe-417c-8278-44962a5a80af-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.229850 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229843    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25b0113e-d86c-464d-89d7-c79c0a4754a9-root\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.229990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.229886    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nzzb\" (UniqueName: \"kubernetes.io/projected/8e637906-06fe-417c-8278-44962a5a80af-kube-api-access-6nzzb\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.230207 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.230180    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e637906-06fe-417c-8278-44962a5a80af-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.232236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.232217    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e637906-06fe-417c-8278-44962a5a80af-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.232339 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.232317    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e637906-06fe-417c-8278-44962a5a80af-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.237851 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.237828    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nzzb\" (UniqueName: \"kubernetes.io/projected/8e637906-06fe-417c-8278-44962a5a80af-kube-api-access-6nzzb\") pod \"openshift-state-metrics-67dcf8c979-ljcqx\" (UID: \"8e637906-06fe-417c-8278-44962a5a80af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.331121 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331086    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-wtmp\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331128    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25b0113e-d86c-464d-89d7-c79c0a4754a9-metrics-client-ca\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331145    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh2rf\" (UniqueName: \"kubernetes.io/projected/25b0113e-d86c-464d-89d7-c79c0a4754a9-kube-api-access-qh2rf\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331164    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331196    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-tls\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331225    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-accelerators-collector-config\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331250    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25b0113e-d86c-464d-89d7-c79c0a4754a9-sys\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331271    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-textfile\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331272    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-wtmp\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331336    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25b0113e-d86c-464d-89d7-c79c0a4754a9-root\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331337    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25b0113e-d86c-464d-89d7-c79c0a4754a9-sys\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331383    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25b0113e-d86c-464d-89d7-c79c0a4754a9-root\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331659 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:12:06.331397    2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 21:12:06.331659 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:12:06.331454    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-tls podName:25b0113e-d86c-464d-89d7-c79c0a4754a9 nodeName:}" failed. No retries permitted until 2026-09-10 21:12:06.831439784 +0000 UTC m=+102.765106581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-tls") pod "node-exporter-rfvqt" (UID: "25b0113e-d86c-464d-89d7-c79c0a4754a9") : secret "node-exporter-tls" not found
Sep 10 21:12:06.331917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331744    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-textfile\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331769    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25b0113e-d86c-464d-89d7-c79c0a4754a9-metrics-client-ca\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.331985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.331916    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-accelerators-collector-config\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.333852 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.333835    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.340083 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.340057    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"
Sep 10 21:12:06.340311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.340293    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh2rf\" (UniqueName: \"kubernetes.io/projected/25b0113e-d86c-464d-89d7-c79c0a4754a9-kube-api-access-qh2rf\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.609036 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.608999    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx"]
Sep 10 21:12:06.611677 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:12:06.611646    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e637906_06fe_417c_8278_44962a5a80af.slice/crio-04f53435f4ac9a1774e2d3a20bd780610fed36374248b803cd3a2c436f4c1245 WatchSource:0}: Error finding container 04f53435f4ac9a1774e2d3a20bd780610fed36374248b803cd3a2c436f4c1245: Status 404 returned error can't find the container with id 04f53435f4ac9a1774e2d3a20bd780610fed36374248b803cd3a2c436f4c1245
Sep 10 21:12:06.836470 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.836431    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-tls\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.838761 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.838735    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25b0113e-d86c-464d-89d7-c79c0a4754a9-node-exporter-tls\") pod \"node-exporter-rfvqt\" (UID: \"25b0113e-d86c-464d-89d7-c79c0a4754a9\") " pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.968989 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:06.968910    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rfvqt"
Sep 10 21:12:06.979922 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:12:06.979893    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b0113e_d86c_464d_89d7_c79c0a4754a9.slice/crio-0b7e78f7194032d43a6c7014a3beb36d65dd1b328b45fa39aceb488eac0d3f51 WatchSource:0}: Error finding container 0b7e78f7194032d43a6c7014a3beb36d65dd1b328b45fa39aceb488eac0d3f51: Status 404 returned error can't find the container with id 0b7e78f7194032d43a6c7014a3beb36d65dd1b328b45fa39aceb488eac0d3f51
Sep 10 21:12:07.120080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.120042    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx" event={"ID":"8e637906-06fe-417c-8278-44962a5a80af","Type":"ContainerStarted","Data":"1782192dce1bc95da9d61fe7ccb2c88d3aa947b44cb7dbc65f59061d06fe109d"}
Sep 10 21:12:07.120506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.120134    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx" event={"ID":"8e637906-06fe-417c-8278-44962a5a80af","Type":"ContainerStarted","Data":"1fcd12b16b412f7f1333a1a86ace73725768cee95287ec80d45bf35ff58a7aa7"}
Sep 10 21:12:07.120506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.120160    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx" event={"ID":"8e637906-06fe-417c-8278-44962a5a80af","Type":"ContainerStarted","Data":"04f53435f4ac9a1774e2d3a20bd780610fed36374248b803cd3a2c436f4c1245"}
Sep 10 21:12:07.121714 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.121685    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfvqt" event={"ID":"25b0113e-d86c-464d-89d7-c79c0a4754a9","Type":"ContainerStarted","Data":"0b7e78f7194032d43a6c7014a3beb36d65dd1b328b45fa39aceb488eac0d3f51"}
Sep 10 21:12:07.247243 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.247166    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:12:07.250658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.250628    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.254638 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.254610    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 21:12:07.254913 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.254891    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 21:12:07.255022 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.254932    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 21:12:07.255085 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.255017    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 21:12:07.255085 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.255043    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 21:12:07.255288 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.255270    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7h96l\""
Sep 10 21:12:07.255360 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.255319    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 21:12:07.255424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.255323    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 21:12:07.255548 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.255526    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 21:12:07.255706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.255688    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 21:12:07.274260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.274224    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:12:07.341335 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341298    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xwbk\" (UniqueName: \"kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-kube-api-access-8xwbk\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341356    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341399    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-out\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341419    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341460    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341474    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341665 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341499    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341665 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341589    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341665 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341635    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341756 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341667    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341756 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341696    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-web-config\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341756 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341720    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.341756 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.341747    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.442994 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.442953    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.442994 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443000    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443021    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443038    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443060    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-web-config\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443074    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443090    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443121    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xwbk\" (UniqueName: \"kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-kube-api-access-8xwbk\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443161    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443241    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-out\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443621 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443268    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443621 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443311    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443621 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443333    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.443621 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:12:07.443411    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-trusted-ca-bundle podName:8595fc83-c3e8-470a-aea0-4d1f200ed2a6 nodeName:}" failed. No retries permitted until 2026-09-10 21:12:07.943390005 +0000 UTC m=+103.877056804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6") : configmap references non-existent config key: ca-bundle.crt
Sep 10 21:12:07.444069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.443864    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.445247 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.444827    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.446296 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.446139    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-out\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.446296 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.446220    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.446296 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.446254    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.446506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.446334    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.446506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.446440    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.446798 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.446753    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.447630 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.447613    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-web-config\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.447806 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.447767    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.448131 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.448116    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.451650 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.451633    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xwbk\" (UniqueName: \"kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-kube-api-access-8xwbk\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.948878 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.948836    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:07.949902 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:07.949861    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:08.161981 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:08.161940    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:12:08.550173 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:08.550142    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:12:08.551890 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:12:08.551861    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8595fc83_c3e8_470a_aea0_4d1f200ed2a6.slice/crio-b550012c886005fe201c962c1dfba80178873b8a65a836c2d11f376191a400a5 WatchSource:0}: Error finding container b550012c886005fe201c962c1dfba80178873b8a65a836c2d11f376191a400a5: Status 404 returned error can't find the container with id b550012c886005fe201c962c1dfba80178873b8a65a836c2d11f376191a400a5
Sep 10 21:12:09.132835 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:09.132797    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx" event={"ID":"8e637906-06fe-417c-8278-44962a5a80af","Type":"ContainerStarted","Data":"b0587e41d994ef1bca711f1c14607b6ca5376e1b1b87483d726bcf0c84962b43"}
Sep 10 21:12:09.134444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:09.134415    2573 generic.go:356] "Generic (PLEG): container finished" podID="25b0113e-d86c-464d-89d7-c79c0a4754a9" containerID="61c28cd93eb5b8ac7c52372ab261ebd17ce86d6ac22f18fdfcc4321361e98477" exitCode=0
Sep 10 21:12:09.134574 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:09.134507    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfvqt" event={"ID":"25b0113e-d86c-464d-89d7-c79c0a4754a9","Type":"ContainerDied","Data":"61c28cd93eb5b8ac7c52372ab261ebd17ce86d6ac22f18fdfcc4321361e98477"}
Sep 10 21:12:09.135870 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:09.135841    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerStarted","Data":"b550012c886005fe201c962c1dfba80178873b8a65a836c2d11f376191a400a5"}
Sep 10 21:12:09.153518 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:09.153467    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljcqx" podStartSLOduration=1.649899915 podStartE2EDuration="3.153446822s" totalImagesPullingTime="1.503546907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:09.151709197 +0000 UTC m=+105.085376011" watchObservedRunningTime="2026-09-10 21:12:09.153446822 +0000 UTC m=+105.087113637"
Sep 10 21:12:10.142304 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:10.142263    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfvqt" event={"ID":"25b0113e-d86c-464d-89d7-c79c0a4754a9","Type":"ContainerStarted","Data":"6b9c0da38c1eba9d5d709226f5730fc5e33b440874521a02c4e3eddf3edcfdf7"}
Sep 10 21:12:10.142724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:10.142320    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfvqt" event={"ID":"25b0113e-d86c-464d-89d7-c79c0a4754a9","Type":"ContainerStarted","Data":"85f83a7630363481554e3179e47f8bd00c0e9b92ecce0866c503c4dffcd99a09"}
Sep 10 21:12:10.167165 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:10.167106    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rfvqt" podStartSLOduration=2.814050764 podStartE2EDuration="4.167088885s" totalImagesPullingTime="1.286153148s" totalInitContainerRuntime="66.884973ms" isStatefulPod=true podCreationTimestamp="2026-09-10 21:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:10.166859448 +0000 UTC m=+106.100526264" watchObservedRunningTime="2026-09-10 21:12:10.167088885 +0000 UTC m=+106.100755699"
Sep 10 21:12:11.147754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:11.147716    2573 generic.go:356] "Generic (PLEG): container finished" podID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerID="87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8" exitCode=0
Sep 10 21:12:11.148180 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:11.147802    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerDied","Data":"87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8"}
Sep 10 21:12:13.161806 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:13.161761    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerStarted","Data":"2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722"}
Sep 10 21:12:13.162236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:13.161892    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerStarted","Data":"6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0"}
Sep 10 21:12:13.162236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:13.161945    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerStarted","Data":"e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f"}
Sep 10 21:12:13.162236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:13.161988    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerStarted","Data":"5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44"}
Sep 10 21:12:13.162236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:13.162027    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerStarted","Data":"fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca"}
Sep 10 21:12:13.399666 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:13.399589    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:12:14.169273 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:14.169241    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerStarted","Data":"e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f"}
Sep 10 21:12:14.197867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:14.197817    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.981137957 podStartE2EDuration="7.197797867s" totalImagesPullingTime="4.203117523s" totalInitContainerRuntime="13.542387ms" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:14.194539212 +0000 UTC m=+110.128206037" watchObservedRunningTime="2026-09-10 21:12:14.197797867 +0000 UTC m=+110.131464673"
Sep 10 21:12:16.061172 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:16.061141    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c8bb7b487-s8pmp"
Sep 10 21:12:16.124144 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:16.124114    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dqff7"
Sep 10 21:12:18.413488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.413445    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-68575978f-cqs5p" podUID="eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" containerName="registry" containerID="cri-o://257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333" gracePeriod=30
Sep 10 21:12:18.662708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.662683    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:12:18.748263 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.748172    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-certificates\") pod \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") "
Sep 10 21:12:18.748263 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.748211    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-ca-trust-extracted\") pod \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") "
Sep 10 21:12:18.748263 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.748238    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-trusted-ca\") pod \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") "
Sep 10 21:12:18.748546 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.748269    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h76fp\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-kube-api-access-h76fp\") pod \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") "
Sep 10 21:12:18.748546 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.748302    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-bound-sa-token\") pod \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") "
Sep 10 21:12:18.748546 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.748327    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-installation-pull-secrets\") pod \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") "
Sep 10 21:12:18.748546 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.748347    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-image-registry-private-configuration\") pod \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") "
Sep 10 21:12:18.748546 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.748385    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls\") pod \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\" (UID: \"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba\") "
Sep 10 21:12:18.748798 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.748664    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:18.748798 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.748729    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:18.750964 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.750935    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:18.751082 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.750983    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-kube-api-access-h76fp" (OuterVolumeSpecName: "kube-api-access-h76fp") pod "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba"). InnerVolumeSpecName "kube-api-access-h76fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:18.751082 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.751057    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:18.751169 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.751101    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:18.751268 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.751251    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:18.759606 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.759576    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" (UID: "eebd3773-65d7-4a74-bc3a-00fc3d8f1fba"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:12:18.849959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.849923    2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-image-registry-private-configuration\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:12:18.849959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.849952    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-tls\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:12:18.849959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.849964    2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-registry-certificates\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:12:18.849959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.849973    2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-ca-trust-extracted\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:12:18.850229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.849982    2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-trusted-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:12:18.850229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.849990    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h76fp\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-kube-api-access-h76fp\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:12:18.850229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.850000    2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-bound-sa-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:12:18.850229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:18.850008    2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba-installation-pull-secrets\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:12:19.190507 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:19.190467    2573 generic.go:356] "Generic (PLEG): container finished" podID="eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" containerID="257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333" exitCode=0
Sep 10 21:12:19.190712 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:19.190516    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68575978f-cqs5p"
Sep 10 21:12:19.190712 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:19.190551    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68575978f-cqs5p" event={"ID":"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba","Type":"ContainerDied","Data":"257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333"}
Sep 10 21:12:19.190712 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:19.190609    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68575978f-cqs5p" event={"ID":"eebd3773-65d7-4a74-bc3a-00fc3d8f1fba","Type":"ContainerDied","Data":"9b01f41066c544d8ce34d3d7848b3926bf7c149ca7294e14e24be5e46cedf2b0"}
Sep 10 21:12:19.190712 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:19.190632    2573 scope.go:117] "RemoveContainer" containerID="257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333"
Sep 10 21:12:19.199724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:19.199706    2573 scope.go:117] "RemoveContainer" containerID="257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333"
Sep 10 21:12:19.200208 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:12:19.200109    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333\": container with ID starting with 257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333 not found: ID does not exist" containerID="257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333"
Sep 10 21:12:19.200208 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:19.200150    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333"} err="failed to get container status \"257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333\": rpc error: code = NotFound desc = could not find container \"257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333\": container with ID starting with 257ef6955a46b799e96efa0940dfb669bc63e6ea66c046aa571a2acf857d7333 not found: ID does not exist"
Sep 10 21:12:19.227247 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:19.227215    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68575978f-cqs5p"]
Sep 10 21:12:19.229966 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:19.229929    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-68575978f-cqs5p"]
Sep 10 21:12:20.677434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:20.677400    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" path="/var/lib/kubelet/pods/eebd3773-65d7-4a74-bc3a-00fc3d8f1fba/volumes"
Sep 10 21:12:34.278991 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:34.278942    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:12:34.281420 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:34.281395    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87136fc5-90a6-4728-9410-a51d6050e497-metrics-certs\") pod \"network-metrics-daemon-vqprq\" (UID: \"87136fc5-90a6-4728-9410-a51d6050e497\") " pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:12:34.287555 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:34.287533    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xghkt\""
Sep 10 21:12:34.296374 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:34.296353    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqprq"
Sep 10 21:12:34.570667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:34.570592    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vqprq"]
Sep 10 21:12:34.573578 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:12:34.573547    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87136fc5_90a6_4728_9410_a51d6050e497.slice/crio-b72cc1483ba1f4705b80947271491f88c6438774794943ccd7b719fc4851f8e8 WatchSource:0}: Error finding container b72cc1483ba1f4705b80947271491f88c6438774794943ccd7b719fc4851f8e8: Status 404 returned error can't find the container with id b72cc1483ba1f4705b80947271491f88c6438774794943ccd7b719fc4851f8e8
Sep 10 21:12:35.249786 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:35.249731    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vqprq" event={"ID":"87136fc5-90a6-4728-9410-a51d6050e497","Type":"ContainerStarted","Data":"b72cc1483ba1f4705b80947271491f88c6438774794943ccd7b719fc4851f8e8"}
Sep 10 21:12:36.255457 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:36.255413    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vqprq" event={"ID":"87136fc5-90a6-4728-9410-a51d6050e497","Type":"ContainerStarted","Data":"efd5150f4f06e2f1342c69f09d71937fcad38cc593eb383a39336cc77af9482f"}
Sep 10 21:12:36.255917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:36.255513    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vqprq" event={"ID":"87136fc5-90a6-4728-9410-a51d6050e497","Type":"ContainerStarted","Data":"858a8979aefbfa6395bcf4fa0fcc83abb987e115f7b5191d6f5361873c5ee6b9"}
Sep 10 21:12:36.276913 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:12:36.276856    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vqprq" podStartSLOduration=131.355001195 podStartE2EDuration="2m12.276838293s" totalImagesPullingTime="921.837098ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:36.275241531 +0000 UTC m=+132.208908346" watchObservedRunningTime="2026-09-10 21:12:36.276838293 +0000 UTC m=+132.210505107"
Sep 10 21:13:26.488179 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:26.488141    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:13:26.488680 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:26.488412    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="alertmanager" containerID="cri-o://fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca" gracePeriod=120
Sep 10 21:13:26.488680 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:26.488433    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy-metric" containerID="cri-o://2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722" gracePeriod=120
Sep 10 21:13:26.488680 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:26.488484    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy-web" containerID="cri-o://e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f" gracePeriod=120
Sep 10 21:13:26.488680 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:26.488526    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy" containerID="cri-o://6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0" gracePeriod=120
Sep 10 21:13:26.488680 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:26.488591    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="config-reloader" containerID="cri-o://5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44" gracePeriod=120
Sep 10 21:13:26.488680 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:26.488589    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="prom-label-proxy" containerID="cri-o://e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f" gracePeriod=120
Sep 10 21:13:27.457628 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.457592    2573 generic.go:356] "Generic (PLEG): container finished" podID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerID="e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f" exitCode=0
Sep 10 21:13:27.457628 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.457618    2573 generic.go:356] "Generic (PLEG): container finished" podID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerID="6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0" exitCode=0
Sep 10 21:13:27.457628 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.457624    2573 generic.go:356] "Generic (PLEG): container finished" podID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerID="5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44" exitCode=0
Sep 10 21:13:27.457628 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.457631    2573 generic.go:356] "Generic (PLEG): container finished" podID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerID="fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca" exitCode=0
Sep 10 21:13:27.457990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.457670    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerDied","Data":"e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f"}
Sep 10 21:13:27.457990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.457807    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerDied","Data":"6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0"}
Sep 10 21:13:27.457990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.457851    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerDied","Data":"5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44"}
Sep 10 21:13:27.457990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.457889    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerDied","Data":"fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca"}
Sep 10 21:13:27.731364 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.731339    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:27.808015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.807981    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-main-tls\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808017    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-volume\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808228 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808061    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808228 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808094    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-metrics-client-ca\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808228 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808130    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808388 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808275    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808388 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808338    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-trusted-ca-bundle\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808388 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808375    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-cluster-tls-config\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808528 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808408    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-tls-assets\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808528 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808444    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-web-config\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808528 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808518    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-out\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808549    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-main-db\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.808676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808597    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xwbk\" (UniqueName: \"kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-kube-api-access-8xwbk\") pod \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\" (UID: \"8595fc83-c3e8-470a-aea0-4d1f200ed2a6\") "
Sep 10 21:13:27.811180 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.808498    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:27.811339 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.809038    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:27.811503 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.811485    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:27.811623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.811579    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:27.811743 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.811642    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:27.811955 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.811920    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-volume" (OuterVolumeSpecName: "config-volume") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:27.811955 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.811943    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:27.813008 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.812972    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:27.813120 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.813056    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:27.813495 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.813466    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-out" (OuterVolumeSpecName: "config-out") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:27.813605 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.813579    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-kube-api-access-8xwbk" (OuterVolumeSpecName: "kube-api-access-8xwbk") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "kube-api-access-8xwbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:27.816700 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.816671    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:27.821973 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.821947    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-web-config" (OuterVolumeSpecName: "web-config") pod "8595fc83-c3e8-470a-aea0-4d1f200ed2a6" (UID: "8595fc83-c3e8-470a-aea0-4d1f200ed2a6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:27.909477 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909432    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xwbk\" (UniqueName: \"kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-kube-api-access-8xwbk\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909477 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909469    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-main-tls\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909477 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909480    2573 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-volume\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909493    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909504    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-metrics-client-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909512    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909522    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909530    2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909539    2573 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-cluster-tls-config\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909546    2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-tls-assets\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909555    2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-web-config\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909563    2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-config-out\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:27.909709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:27.909571    2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8595fc83-c3e8-470a-aea0-4d1f200ed2a6-alertmanager-main-db\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:13:28.465459 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.465424    2573 generic.go:356] "Generic (PLEG): container finished" podID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerID="2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722" exitCode=0
Sep 10 21:13:28.465459 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.465448    2573 generic.go:356] "Generic (PLEG): container finished" podID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerID="e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f" exitCode=0
Sep 10 21:13:28.465714 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.465474    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerDied","Data":"2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722"}
Sep 10 21:13:28.465714 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.465520    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.465714 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.465531    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerDied","Data":"e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f"}
Sep 10 21:13:28.465714 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.465558    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8595fc83-c3e8-470a-aea0-4d1f200ed2a6","Type":"ContainerDied","Data":"b550012c886005fe201c962c1dfba80178873b8a65a836c2d11f376191a400a5"}
Sep 10 21:13:28.465714 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.465588    2573 scope.go:117] "RemoveContainer" containerID="e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f"
Sep 10 21:13:28.475634 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.475594    2573 scope.go:117] "RemoveContainer" containerID="2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722"
Sep 10 21:13:28.483541 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.483525    2573 scope.go:117] "RemoveContainer" containerID="6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0"
Sep 10 21:13:28.490723 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.490699    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:13:28.491047 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.491030    2573 scope.go:117] "RemoveContainer" containerID="e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f"
Sep 10 21:13:28.495512 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.495491    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:13:28.498503 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.498488    2573 scope.go:117] "RemoveContainer" containerID="5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44"
Sep 10 21:13:28.505497 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.505478    2573 scope.go:117] "RemoveContainer" containerID="fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca"
Sep 10 21:13:28.512503 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.512480    2573 scope.go:117] "RemoveContainer" containerID="87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8"
Sep 10 21:13:28.519295 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.519278    2573 scope.go:117] "RemoveContainer" containerID="e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f"
Sep 10 21:13:28.519560 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:13:28.519540    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f\": container with ID starting with e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f not found: ID does not exist" containerID="e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f"
Sep 10 21:13:28.519614 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.519581    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f"} err="failed to get container status \"e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f\": rpc error: code = NotFound desc = could not find container \"e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f\": container with ID starting with e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f not found: ID does not exist"
Sep 10 21:13:28.519614 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.519600    2573 scope.go:117] "RemoveContainer" containerID="2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722"
Sep 10 21:13:28.519845 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:13:28.519830    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722\": container with ID starting with 2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722 not found: ID does not exist" containerID="2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722"
Sep 10 21:13:28.519891 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.519849    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722"} err="failed to get container status \"2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722\": rpc error: code = NotFound desc = could not find container \"2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722\": container with ID starting with 2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722 not found: ID does not exist"
Sep 10 21:13:28.519891 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.519862    2573 scope.go:117] "RemoveContainer" containerID="6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0"
Sep 10 21:13:28.520088 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:13:28.520074    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0\": container with ID starting with 6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0 not found: ID does not exist" containerID="6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0"
Sep 10 21:13:28.520130 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.520090    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0"} err="failed to get container status \"6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0\": rpc error: code = NotFound desc = could not find container \"6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0\": container with ID starting with 6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0 not found: ID does not exist"
Sep 10 21:13:28.520130 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.520100    2573 scope.go:117] "RemoveContainer" containerID="e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f"
Sep 10 21:13:28.520341 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:13:28.520324    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f\": container with ID starting with e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f not found: ID does not exist" containerID="e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f"
Sep 10 21:13:28.520381 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.520343    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f"} err="failed to get container status \"e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f\": rpc error: code = NotFound desc = could not find container \"e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f\": container with ID starting with e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f not found: ID does not exist"
Sep 10 21:13:28.520381 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.520354    2573 scope.go:117] "RemoveContainer" containerID="5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44"
Sep 10 21:13:28.520538 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:13:28.520524    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44\": container with ID starting with 5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44 not found: ID does not exist" containerID="5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44"
Sep 10 21:13:28.520579 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.520540    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44"} err="failed to get container status \"5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44\": rpc error: code = NotFound desc = could not find container \"5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44\": container with ID starting with 5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44 not found: ID does not exist"
Sep 10 21:13:28.520579 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.520551    2573 scope.go:117] "RemoveContainer" containerID="fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca"
Sep 10 21:13:28.520749 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:13:28.520733    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca\": container with ID starting with fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca not found: ID does not exist" containerID="fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca"
Sep 10 21:13:28.520838 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.520751    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca"} err="failed to get container status \"fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca\": rpc error: code = NotFound desc = could not find container \"fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca\": container with ID starting with fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca not found: ID does not exist"
Sep 10 21:13:28.520838 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.520764    2573 scope.go:117] "RemoveContainer" containerID="87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8"
Sep 10 21:13:28.520992 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:13:28.520970    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8\": container with ID starting with 87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8 not found: ID does not exist" containerID="87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8"
Sep 10 21:13:28.521033 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.520999    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8"} err="failed to get container status \"87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8\": rpc error: code = NotFound desc = could not find container \"87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8\": container with ID starting with 87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8 not found: ID does not exist"
Sep 10 21:13:28.521033 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.521014    2573 scope.go:117] "RemoveContainer" containerID="e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f"
Sep 10 21:13:28.521221 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.521205    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f"} err="failed to get container status \"e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f\": rpc error: code = NotFound desc = could not find container \"e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f\": container with ID starting with e86c434401f5d351a49b3518593d40cf2711f91e6ccabe522400ba0a463e953f not found: ID does not exist"
Sep 10 21:13:28.521377 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.521222    2573 scope.go:117] "RemoveContainer" containerID="2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722"
Sep 10 21:13:28.521507 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.521488    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722"} err="failed to get container status \"2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722\": rpc error: code = NotFound desc = could not find container \"2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722\": container with ID starting with 2d9247c2bf74b3896d4952b45ddaba71f821ff7940484c2c6df5f0581664a722 not found: ID does not exist"
Sep 10 21:13:28.521582 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.521507    2573 scope.go:117] "RemoveContainer" containerID="6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0"
Sep 10 21:13:28.521826 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.521799    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0"} err="failed to get container status \"6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0\": rpc error: code = NotFound desc = could not find container \"6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0\": container with ID starting with 6707a2bef78da4eb9e740c0101dacbfc8d19d77b83f37991906715c6f4a6fcd0 not found: ID does not exist"
Sep 10 21:13:28.521826 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.521825    2573 scope.go:117] "RemoveContainer" containerID="e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f"
Sep 10 21:13:28.522254 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.522230    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f"} err="failed to get container status \"e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f\": rpc error: code = NotFound desc = could not find container \"e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f\": container with ID starting with e547e8e6fb12c8744c8181c2019b10de8c5eaf9eaba6ab480d18dc8412d10a6f not found: ID does not exist"
Sep 10 21:13:28.522350 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.522255    2573 scope.go:117] "RemoveContainer" containerID="5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44"
Sep 10 21:13:28.522637 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.522530    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44"} err="failed to get container status \"5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44\": rpc error: code = NotFound desc = could not find container \"5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44\": container with ID starting with 5eaa95169008dfea947c56d082b94aa3f9c5699aa3df1683e9917b83c710aa44 not found: ID does not exist"
Sep 10 21:13:28.522637 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.522555    2573 scope.go:117] "RemoveContainer" containerID="fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca"
Sep 10 21:13:28.522898 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.522878    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca"} err="failed to get container status \"fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca\": rpc error: code = NotFound desc = could not find container \"fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca\": container with ID starting with fe36d9fa82c9f40fde79d528950ff4b7d51a87b820a7720c87b3b318c47313ca not found: ID does not exist"
Sep 10 21:13:28.522898 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.522898    2573 scope.go:117] "RemoveContainer" containerID="87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8"
Sep 10 21:13:28.523179 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523160    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8"} err="failed to get container status \"87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8\": rpc error: code = NotFound desc = could not find container \"87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8\": container with ID starting with 87530aa509d8181e6e484bbed34fca4c045d59f7270729183228a71295fe34c8 not found: ID does not exist"
Sep 10 21:13:28.523286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523269    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:13:28.523519 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523509    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy-web"
Sep 10 21:13:28.523553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523521    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy-web"
Sep 10 21:13:28.523553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523533    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy-metric"
Sep 10 21:13:28.523553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523539    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy-metric"
Sep 10 21:13:28.523553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523547    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="alertmanager"
Sep 10 21:13:28.523553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523552    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="alertmanager"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523560    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="prom-label-proxy"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523574    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="prom-label-proxy"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523585    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="config-reloader"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523591    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="config-reloader"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523596    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" containerName="registry"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523601    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" containerName="registry"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523608    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="init-config-reloader"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523613    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="init-config-reloader"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523619    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523625    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523664    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy-metric"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523672    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy-web"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523678    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="kube-rbac-proxy"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523683    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="eebd3773-65d7-4a74-bc3a-00fc3d8f1fba" containerName="registry"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523688    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="prom-label-proxy"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523695    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="config-reloader"
Sep 10 21:13:28.523706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.523701    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" containerName="alertmanager"
Sep 10 21:13:28.527984 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.527966    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.530246 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.530215    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 21:13:28.530246 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.530229    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 21:13:28.530402 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.530262    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7h96l\""
Sep 10 21:13:28.530402 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.530272    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 21:13:28.530402 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.530216    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 21:13:28.530402 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.530337    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 21:13:28.530708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.530685    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 21:13:28.533309 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.531315    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 21:13:28.533309 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.531538    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 21:13:28.535760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.535744    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 21:13:28.544753 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.544724    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:13:28.615354 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615318    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0939e986-4979-4c34-a308-18c22729f75d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615354 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615354    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0939e986-4979-4c34-a308-18c22729f75d-config-out\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615547 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615374    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0939e986-4979-4c34-a308-18c22729f75d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615547 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615402    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615547 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615433    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0939e986-4979-4c34-a308-18c22729f75d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615547 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615470    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615675 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615539    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0939e986-4979-4c34-a308-18c22729f75d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615675 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615577    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615675 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615602    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-web-config\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615675 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615664    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615821 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615679    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615821 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615700    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7rml\" (UniqueName: \"kubernetes.io/projected/0939e986-4979-4c34-a308-18c22729f75d-kube-api-access-x7rml\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.615821 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.615716    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.677485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.677449    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8595fc83-c3e8-470a-aea0-4d1f200ed2a6" path="/var/lib/kubelet/pods/8595fc83-c3e8-470a-aea0-4d1f200ed2a6/volumes"
Sep 10 21:13:28.716119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716036    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0939e986-4979-4c34-a308-18c22729f75d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716098    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716125    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0939e986-4979-4c34-a308-18c22729f75d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716176    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716213    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0939e986-4979-4c34-a308-18c22729f75d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716247    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716283    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-web-config\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716320    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716348    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716384    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7rml\" (UniqueName: \"kubernetes.io/projected/0939e986-4979-4c34-a308-18c22729f75d-kube-api-access-x7rml\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716410    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716444    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0939e986-4979-4c34-a308-18c22729f75d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716663 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716466    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0939e986-4979-4c34-a308-18c22729f75d-config-out\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.716713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.716682    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0939e986-4979-4c34-a308-18c22729f75d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.717639 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.717321    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0939e986-4979-4c34-a308-18c22729f75d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.718142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.718099    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0939e986-4979-4c34-a308-18c22729f75d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.719511 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.719466    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0939e986-4979-4c34-a308-18c22729f75d-config-out\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.719613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.719559    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-web-config\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.719613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.719588    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.720019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.719987    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.720097 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.720082    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0939e986-4979-4c34-a308-18c22729f75d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.720441 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.720410    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.720536 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.720483    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.720597 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.720567    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.720652 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.720640    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0939e986-4979-4c34-a308-18c22729f75d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.726012 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.725992    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7rml\" (UniqueName: \"kubernetes.io/projected/0939e986-4979-4c34-a308-18c22729f75d-kube-api-access-x7rml\") pod \"alertmanager-main-0\" (UID: \"0939e986-4979-4c34-a308-18c22729f75d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:28.837754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:28.837717    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:13:29.112500 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:29.112359    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:13:29.115290 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:13:29.115264    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0939e986_4979_4c34_a308_18c22729f75d.slice/crio-bcf6b08aa9a8bf3de2c6bfe6e31b2eb0749044711cc0ad9556b97eb8af9f4ae8 WatchSource:0}: Error finding container bcf6b08aa9a8bf3de2c6bfe6e31b2eb0749044711cc0ad9556b97eb8af9f4ae8: Status 404 returned error can't find the container with id bcf6b08aa9a8bf3de2c6bfe6e31b2eb0749044711cc0ad9556b97eb8af9f4ae8
Sep 10 21:13:29.470934 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:29.470841    2573 generic.go:356] "Generic (PLEG): container finished" podID="0939e986-4979-4c34-a308-18c22729f75d" containerID="7946d28acefac8e42662e75276217a683e48c45b7ad5cd71a31aa771b71d6c8c" exitCode=0
Sep 10 21:13:29.470934 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:29.470886    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0939e986-4979-4c34-a308-18c22729f75d","Type":"ContainerDied","Data":"7946d28acefac8e42662e75276217a683e48c45b7ad5cd71a31aa771b71d6c8c"}
Sep 10 21:13:29.471160 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:29.470942    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0939e986-4979-4c34-a308-18c22729f75d","Type":"ContainerStarted","Data":"bcf6b08aa9a8bf3de2c6bfe6e31b2eb0749044711cc0ad9556b97eb8af9f4ae8"}
Sep 10 21:13:30.480321 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.480289    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0939e986-4979-4c34-a308-18c22729f75d","Type":"ContainerStarted","Data":"e47bcbf389ca08335cbf254a28550a6c36cfe146cc86d9739b9f0c7cb8c1d408"}
Sep 10 21:13:30.480770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.480377    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0939e986-4979-4c34-a308-18c22729f75d","Type":"ContainerStarted","Data":"1b1ee09c8787cca711daa7b4a980d3e7c18228edc296d540ebab28ec3f86b996"}
Sep 10 21:13:30.480770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.480406    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0939e986-4979-4c34-a308-18c22729f75d","Type":"ContainerStarted","Data":"8f13bab61085f5a2783e6f2f0722cc87fe951ae05a4c266b2a81d27ac1ff1284"}
Sep 10 21:13:30.480770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.480429    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0939e986-4979-4c34-a308-18c22729f75d","Type":"ContainerStarted","Data":"8a7754ba171eba075c8d820f3dfbf12ae37afb23c1aecb0503d88a946b3a2851"}
Sep 10 21:13:30.480770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.480453    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0939e986-4979-4c34-a308-18c22729f75d","Type":"ContainerStarted","Data":"ff8fed40ae3794f438401e67e359a64b656647da5979219fca84091b533425c7"}
Sep 10 21:13:30.480770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.480487    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0939e986-4979-4c34-a308-18c22729f75d","Type":"ContainerStarted","Data":"71f0c1fd293d7104e4878bee49aa35060435608aba418bfeca0ed8b5f4ac4f76"}
Sep 10 21:13:30.503502 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.503454    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-774d68476c-v4dqc"]
Sep 10 21:13:30.506584 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.506563    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.509619 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.509579    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 21:13:30.509854 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.509833    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 21:13:30.510181 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.510157    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 21:13:30.510262 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.510178    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 21:13:30.510414 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.510398    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 21:13:30.510476 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.510459    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jnj4k\""
Sep 10 21:13:30.515175 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.515121    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.497592335 podStartE2EDuration="2.515103305s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.51097ms" isStatefulPod=false podCreationTimestamp="2026-09-10 21:13:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:13:30.514423123 +0000 UTC m=+186.448089978" watchObservedRunningTime="2026-09-10 21:13:30.515103305 +0000 UTC m=+186.448770124"
Sep 10 21:13:30.517162 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.517141    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 21:13:30.529546 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.529465    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-774d68476c-v4dqc"]
Sep 10 21:13:30.530436 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.530225    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-telemeter-client-tls\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.530436 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.530272    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-metrics-client-ca\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.530436 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.530320    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-serving-certs-ca-bundle\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.530436 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.530391    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-telemeter-trusted-ca-bundle\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.530739 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.530443    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-secret-telemeter-client\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.530739 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.530468    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-federate-client-tls\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.530739 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.530495    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72b6x\" (UniqueName: \"kubernetes.io/projected/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-kube-api-access-72b6x\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.530909 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.530876    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.631843 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.631805    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-telemeter-trusted-ca-bundle\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.631843 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.631852    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-secret-telemeter-client\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.632057 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.631874    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-federate-client-tls\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.632057 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.631922    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72b6x\" (UniqueName: \"kubernetes.io/projected/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-kube-api-access-72b6x\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.632057 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.631953    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.632057 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.632031    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-telemeter-client-tls\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.632579 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.632058    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-metrics-client-ca\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.632579 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.632226    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-serving-certs-ca-bundle\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.632756 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.632734    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-metrics-client-ca\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.632893 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.632870    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-telemeter-trusted-ca-bundle\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.632978 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.632907    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-serving-certs-ca-bundle\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.634603 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.634577    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-federate-client-tls\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.634715 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.634658    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-secret-telemeter-client\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.634756 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.634723    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.634819 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.634772    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-telemeter-client-tls\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.643809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.642078    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72b6x\" (UniqueName: \"kubernetes.io/projected/a5f04b3f-c716-4d0f-89d0-7a81d7cab965-kube-api-access-72b6x\") pod \"telemeter-client-774d68476c-v4dqc\" (UID: \"a5f04b3f-c716-4d0f-89d0-7a81d7cab965\") " pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:30.818425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:30.818340    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc"
Sep 10 21:13:31.111896 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:31.111861    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-774d68476c-v4dqc"]
Sep 10 21:13:31.486163 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:31.486069    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc" event={"ID":"a5f04b3f-c716-4d0f-89d0-7a81d7cab965","Type":"ContainerStarted","Data":"5c6a63b095c51795f6a630f816a026c69e19333c5f3ede142402ede1dc861e9c"}
Sep 10 21:13:33.498161 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:33.498119    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc" event={"ID":"a5f04b3f-c716-4d0f-89d0-7a81d7cab965","Type":"ContainerStarted","Data":"3c9b454ad2c12dbb9a8b07a160f7bc4461d92a346cb7d96706c53fec5008e3d9"}
Sep 10 21:13:33.498612 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:33.498187    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc" event={"ID":"a5f04b3f-c716-4d0f-89d0-7a81d7cab965","Type":"ContainerStarted","Data":"c24b58856dfec70265a9bda5ac1e86bbc465de58f0fae5da7e8aae592a801409"}
Sep 10 21:13:33.498612 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:33.498217    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc" event={"ID":"a5f04b3f-c716-4d0f-89d0-7a81d7cab965","Type":"ContainerStarted","Data":"dc7bd053368b3a9d1ee2bfb638735a40f54e0795078e4d3dc5a3bcf8dac0cd7a"}
Sep 10 21:13:33.518929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:13:33.518883    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-774d68476c-v4dqc" podStartSLOduration=1.676466774 podStartE2EDuration="3.51886461s" totalImagesPullingTime="1.842397836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:13:33.518838481 +0000 UTC m=+189.452505297" watchObservedRunningTime="2026-09-10 21:13:33.51886461 +0000 UTC m=+189.452531423"
Sep 10 21:15:24.431534 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:24.431500    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:15:24.437719 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:24.437690    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:15:24.440623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:24.440601    2573 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 21:15:25.938172 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:25.938136    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"]
Sep 10 21:15:25.941486 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:25.941465    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:25.943827 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:25.943800    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 21:15:25.944301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:25.944286    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rpb92\""
Sep 10 21:15:25.944301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:25.944293    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 21:15:25.952872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:25.952744    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"]
Sep 10 21:15:25.957715 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:25.957686    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6lv6\" (UniqueName: \"kubernetes.io/projected/594e04a1-38b8-4f0e-833a-551c689f8a88-kube-api-access-g6lv6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:25.957837 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:25.957801    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:25.957880 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:25.957850    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:26.058771 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:26.058728    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:26.058985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:26.058834    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6lv6\" (UniqueName: \"kubernetes.io/projected/594e04a1-38b8-4f0e-833a-551c689f8a88-kube-api-access-g6lv6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:26.058985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:26.058892    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:26.059221 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:26.059196    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:26.059292 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:26.059239    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:26.067339 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:26.067305    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6lv6\" (UniqueName: \"kubernetes.io/projected/594e04a1-38b8-4f0e-833a-551c689f8a88-kube-api-access-g6lv6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:26.250198 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:26.250106    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:26.521328 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:26.521243    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"]
Sep 10 21:15:26.524313 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:15:26.524287    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod594e04a1_38b8_4f0e_833a_551c689f8a88.slice/crio-357d99e8d9e15c7c4f79faddf2b9fe8799270cdff52afc518161309b01a74f08 WatchSource:0}: Error finding container 357d99e8d9e15c7c4f79faddf2b9fe8799270cdff52afc518161309b01a74f08: Status 404 returned error can't find the container with id 357d99e8d9e15c7c4f79faddf2b9fe8799270cdff52afc518161309b01a74f08
Sep 10 21:15:26.526347 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:26.526326    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:15:26.915545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:26.915507    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8" event={"ID":"594e04a1-38b8-4f0e-833a-551c689f8a88","Type":"ContainerStarted","Data":"357d99e8d9e15c7c4f79faddf2b9fe8799270cdff52afc518161309b01a74f08"}
Sep 10 21:15:33.947069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:33.947030    2573 generic.go:356] "Generic (PLEG): container finished" podID="594e04a1-38b8-4f0e-833a-551c689f8a88" containerID="59a2666106e3c588d3ce9087d98bd2c86576aebda2ad0cef03729e46aba51c28" exitCode=0
Sep 10 21:15:33.947422 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:33.947122    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8" event={"ID":"594e04a1-38b8-4f0e-833a-551c689f8a88","Type":"ContainerDied","Data":"59a2666106e3c588d3ce9087d98bd2c86576aebda2ad0cef03729e46aba51c28"}
Sep 10 21:15:36.961016 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:36.960980    2573 generic.go:356] "Generic (PLEG): container finished" podID="594e04a1-38b8-4f0e-833a-551c689f8a88" containerID="113cfd316150807b8e71be056818d2f454e6cc2c56d92f7acfbdb22caf4b1b21" exitCode=0
Sep 10 21:15:36.961420 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:36.961071    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8" event={"ID":"594e04a1-38b8-4f0e-833a-551c689f8a88","Type":"ContainerDied","Data":"113cfd316150807b8e71be056818d2f454e6cc2c56d92f7acfbdb22caf4b1b21"}
Sep 10 21:15:43.993917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:43.993884    2573 generic.go:356] "Generic (PLEG): container finished" podID="594e04a1-38b8-4f0e-833a-551c689f8a88" containerID="d6c819b7290e7bb7ed344e5665e92638075410afe82543cbe46192d9b5899e1f" exitCode=0
Sep 10 21:15:43.994278 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:43.993973    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8" event={"ID":"594e04a1-38b8-4f0e-833a-551c689f8a88","Type":"ContainerDied","Data":"d6c819b7290e7bb7ed344e5665e92638075410afe82543cbe46192d9b5899e1f"}
Sep 10 21:15:45.117875 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:45.117848    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:45.237375 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:45.237334    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-bundle\") pod \"594e04a1-38b8-4f0e-833a-551c689f8a88\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") "
Sep 10 21:15:45.237375 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:45.237381    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-util\") pod \"594e04a1-38b8-4f0e-833a-551c689f8a88\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") "
Sep 10 21:15:45.237593 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:45.237416    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6lv6\" (UniqueName: \"kubernetes.io/projected/594e04a1-38b8-4f0e-833a-551c689f8a88-kube-api-access-g6lv6\") pod \"594e04a1-38b8-4f0e-833a-551c689f8a88\" (UID: \"594e04a1-38b8-4f0e-833a-551c689f8a88\") "
Sep 10 21:15:45.238023 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:45.237997    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-bundle" (OuterVolumeSpecName: "bundle") pod "594e04a1-38b8-4f0e-833a-551c689f8a88" (UID: "594e04a1-38b8-4f0e-833a-551c689f8a88"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:15:45.239762 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:45.239733    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/594e04a1-38b8-4f0e-833a-551c689f8a88-kube-api-access-g6lv6" (OuterVolumeSpecName: "kube-api-access-g6lv6") pod "594e04a1-38b8-4f0e-833a-551c689f8a88" (UID: "594e04a1-38b8-4f0e-833a-551c689f8a88"). InnerVolumeSpecName "kube-api-access-g6lv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:15:45.242281 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:45.242260    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-util" (OuterVolumeSpecName: "util") pod "594e04a1-38b8-4f0e-833a-551c689f8a88" (UID: "594e04a1-38b8-4f0e-833a-551c689f8a88"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:15:45.338945 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:45.338911    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-bundle\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:15:45.338945 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:45.338943    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/594e04a1-38b8-4f0e-833a-551c689f8a88-util\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:15:45.339120 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:45.338953    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6lv6\" (UniqueName: \"kubernetes.io/projected/594e04a1-38b8-4f0e-833a-551c689f8a88-kube-api-access-g6lv6\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:15:46.004062 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:46.004025    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8" event={"ID":"594e04a1-38b8-4f0e-833a-551c689f8a88","Type":"ContainerDied","Data":"357d99e8d9e15c7c4f79faddf2b9fe8799270cdff52afc518161309b01a74f08"}
Sep 10 21:15:46.004226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:46.004068    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402qgg8"
Sep 10 21:15:46.004226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:46.004075    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="357d99e8d9e15c7c4f79faddf2b9fe8799270cdff52afc518161309b01a74f08"
Sep 10 21:15:53.918631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.918594    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"]
Sep 10 21:15:53.919046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.918903    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="594e04a1-38b8-4f0e-833a-551c689f8a88" containerName="extract"
Sep 10 21:15:53.919046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.918914    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="594e04a1-38b8-4f0e-833a-551c689f8a88" containerName="extract"
Sep 10 21:15:53.919046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.918930    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="594e04a1-38b8-4f0e-833a-551c689f8a88" containerName="util"
Sep 10 21:15:53.919046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.918935    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="594e04a1-38b8-4f0e-833a-551c689f8a88" containerName="util"
Sep 10 21:15:53.919046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.918946    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="594e04a1-38b8-4f0e-833a-551c689f8a88" containerName="pull"
Sep 10 21:15:53.919046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.918952    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="594e04a1-38b8-4f0e-833a-551c689f8a88" containerName="pull"
Sep 10 21:15:53.919046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.918994    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="594e04a1-38b8-4f0e-833a-551c689f8a88" containerName="extract"
Sep 10 21:15:53.920919 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.920903    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"
Sep 10 21:15:53.923285 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.923265    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 10 21:15:53.923436 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.923318    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-g8nn8\""
Sep 10 21:15:53.923436 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.923373    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:15:53.937066 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:53.937036    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"]
Sep 10 21:15:54.009808 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:54.009741    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f283997-7950-4a10-87d9-d779a85dd7eb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rj2nn\" (UID: \"4f283997-7950-4a10-87d9-d779a85dd7eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"
Sep 10 21:15:54.009990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:54.009880    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czhzl\" (UniqueName: \"kubernetes.io/projected/4f283997-7950-4a10-87d9-d779a85dd7eb-kube-api-access-czhzl\") pod \"cert-manager-operator-controller-manager-55c444bf86-rj2nn\" (UID: \"4f283997-7950-4a10-87d9-d779a85dd7eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"
Sep 10 21:15:54.110900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:54.110859    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czhzl\" (UniqueName: \"kubernetes.io/projected/4f283997-7950-4a10-87d9-d779a85dd7eb-kube-api-access-czhzl\") pod \"cert-manager-operator-controller-manager-55c444bf86-rj2nn\" (UID: \"4f283997-7950-4a10-87d9-d779a85dd7eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"
Sep 10 21:15:54.111102 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:54.110927    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f283997-7950-4a10-87d9-d779a85dd7eb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rj2nn\" (UID: \"4f283997-7950-4a10-87d9-d779a85dd7eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"
Sep 10 21:15:54.111278 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:54.111260    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f283997-7950-4a10-87d9-d779a85dd7eb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rj2nn\" (UID: \"4f283997-7950-4a10-87d9-d779a85dd7eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"
Sep 10 21:15:54.119210 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:54.119187    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czhzl\" (UniqueName: \"kubernetes.io/projected/4f283997-7950-4a10-87d9-d779a85dd7eb-kube-api-access-czhzl\") pod \"cert-manager-operator-controller-manager-55c444bf86-rj2nn\" (UID: \"4f283997-7950-4a10-87d9-d779a85dd7eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"
Sep 10 21:15:54.230937 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:54.230851    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"
Sep 10 21:15:54.514874 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:54.514795    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn"]
Sep 10 21:15:54.518432 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:15:54.518402    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f283997_7950_4a10_87d9_d779a85dd7eb.slice/crio-e380fd4c16bf199691b896782bb068ec4064e92370e6781af6151fb5fb7ad463 WatchSource:0}: Error finding container e380fd4c16bf199691b896782bb068ec4064e92370e6781af6151fb5fb7ad463: Status 404 returned error can't find the container with id e380fd4c16bf199691b896782bb068ec4064e92370e6781af6151fb5fb7ad463
Sep 10 21:15:55.042633 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:55.042596    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn" event={"ID":"4f283997-7950-4a10-87d9-d779a85dd7eb","Type":"ContainerStarted","Data":"e380fd4c16bf199691b896782bb068ec4064e92370e6781af6151fb5fb7ad463"}
Sep 10 21:15:57.053260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:57.053222    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn" event={"ID":"4f283997-7950-4a10-87d9-d779a85dd7eb","Type":"ContainerStarted","Data":"7de2771d06dc9174a6fd14a8282c6ac29efd6e371b49f70833cae6de3c74e496"}
Sep 10 21:15:57.073335 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:57.073276    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rj2nn" podStartSLOduration=2.5810669710000003 podStartE2EDuration="4.073259378s" totalImagesPullingTime="1.492192407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:57.072929685 +0000 UTC m=+333.006596504" watchObservedRunningTime="2026-09-10 21:15:57.073259378 +0000 UTC m=+333.006926193"
Sep 10 21:15:58.029107 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.029069    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"]
Sep 10 21:15:58.031854 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.031837    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.033838 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.033813    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 21:15:58.033978 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.033840    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 21:15:58.034330 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.034315    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rpb92\""
Sep 10 21:15:58.041594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.041569    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"]
Sep 10 21:15:58.142605 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.142568    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.142605 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.142609    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.143170 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.142634    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp6wx\" (UniqueName: \"kubernetes.io/projected/d4a94661-aef3-4bc5-93f1-912309ffee46-kube-api-access-wp6wx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.244104 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.244070    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.244104 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.244107    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.244326 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.244136    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp6wx\" (UniqueName: \"kubernetes.io/projected/d4a94661-aef3-4bc5-93f1-912309ffee46-kube-api-access-wp6wx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.244485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.244466    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.244520 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.244502    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.280982 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.280911    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp6wx\" (UniqueName: \"kubernetes.io/projected/d4a94661-aef3-4bc5-93f1-912309ffee46-kube-api-access-wp6wx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.340691 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.340649    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:15:58.609349 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:58.609321    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"]
Sep 10 21:15:58.612004 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:15:58.611977    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a94661_aef3_4bc5_93f1_912309ffee46.slice/crio-50b8ae8baca04f6057e3cefd789a1ae380670fa050009a4496cff7ba59ad2576 WatchSource:0}: Error finding container 50b8ae8baca04f6057e3cefd789a1ae380670fa050009a4496cff7ba59ad2576: Status 404 returned error can't find the container with id 50b8ae8baca04f6057e3cefd789a1ae380670fa050009a4496cff7ba59ad2576
Sep 10 21:15:59.063261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:59.063166    2573 generic.go:356] "Generic (PLEG): container finished" podID="d4a94661-aef3-4bc5-93f1-912309ffee46" containerID="38ca30ba1bc55457acd3f03853b501e15e8c42d8cbba5b8bf17f4c7c462fde14" exitCode=0
Sep 10 21:15:59.063261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:59.063219    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22" event={"ID":"d4a94661-aef3-4bc5-93f1-912309ffee46","Type":"ContainerDied","Data":"38ca30ba1bc55457acd3f03853b501e15e8c42d8cbba5b8bf17f4c7c462fde14"}
Sep 10 21:15:59.063498 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:15:59.063285    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22" event={"ID":"d4a94661-aef3-4bc5-93f1-912309ffee46","Type":"ContainerStarted","Data":"50b8ae8baca04f6057e3cefd789a1ae380670fa050009a4496cff7ba59ad2576"}
Sep 10 21:16:01.935356 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:01.935319    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"]
Sep 10 21:16:01.937683 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:01.937667    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"
Sep 10 21:16:01.939827 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:01.939795    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-v2scf\""
Sep 10 21:16:01.939960 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:01.939804    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 21:16:01.940404 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:01.940386    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 21:16:01.953078 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:01.950849    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"]
Sep 10 21:16:02.077369 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:02.077331    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsjmm\" (UniqueName: \"kubernetes.io/projected/6ae7e5c3-2b67-47b0-9506-63f06f0993a1-kube-api-access-nsjmm\") pod \"cert-manager-cainjector-559cdd8c87-zn6ss\" (UID: \"6ae7e5c3-2b67-47b0-9506-63f06f0993a1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"
Sep 10 21:16:02.077545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:02.077404    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6ae7e5c3-2b67-47b0-9506-63f06f0993a1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zn6ss\" (UID: \"6ae7e5c3-2b67-47b0-9506-63f06f0993a1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"
Sep 10 21:16:02.082729 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:02.082698    2573 generic.go:356] "Generic (PLEG): container finished" podID="d4a94661-aef3-4bc5-93f1-912309ffee46" containerID="25c1e7d11edabd3c52475d64902dde37cbed91ae8f85fd4c7bd95aeb2f7fcbf1" exitCode=0
Sep 10 21:16:02.082886 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:02.082791    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22" event={"ID":"d4a94661-aef3-4bc5-93f1-912309ffee46","Type":"ContainerDied","Data":"25c1e7d11edabd3c52475d64902dde37cbed91ae8f85fd4c7bd95aeb2f7fcbf1"}
Sep 10 21:16:02.178371 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:02.178335    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsjmm\" (UniqueName: \"kubernetes.io/projected/6ae7e5c3-2b67-47b0-9506-63f06f0993a1-kube-api-access-nsjmm\") pod \"cert-manager-cainjector-559cdd8c87-zn6ss\" (UID: \"6ae7e5c3-2b67-47b0-9506-63f06f0993a1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"
Sep 10 21:16:02.178515 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:02.178387    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6ae7e5c3-2b67-47b0-9506-63f06f0993a1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zn6ss\" (UID: \"6ae7e5c3-2b67-47b0-9506-63f06f0993a1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"
Sep 10 21:16:02.188457 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:02.188356    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsjmm\" (UniqueName: \"kubernetes.io/projected/6ae7e5c3-2b67-47b0-9506-63f06f0993a1-kube-api-access-nsjmm\") pod \"cert-manager-cainjector-559cdd8c87-zn6ss\" (UID: \"6ae7e5c3-2b67-47b0-9506-63f06f0993a1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"
Sep 10 21:16:02.188740 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:02.188700    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6ae7e5c3-2b67-47b0-9506-63f06f0993a1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zn6ss\" (UID: \"6ae7e5c3-2b67-47b0-9506-63f06f0993a1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"
Sep 10 21:16:02.262235 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:02.262199    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"
Sep 10 21:16:02.533052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:02.533025    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss"]
Sep 10 21:16:02.535580 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:16:02.535547    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ae7e5c3_2b67_47b0_9506_63f06f0993a1.slice/crio-dfd6bc2891ed37ffab84151f440b1a56ed6456a32e21850dc75ed9c5410befa7 WatchSource:0}: Error finding container dfd6bc2891ed37ffab84151f440b1a56ed6456a32e21850dc75ed9c5410befa7: Status 404 returned error can't find the container with id dfd6bc2891ed37ffab84151f440b1a56ed6456a32e21850dc75ed9c5410befa7
Sep 10 21:16:03.089021 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:03.088979    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss" event={"ID":"6ae7e5c3-2b67-47b0-9506-63f06f0993a1","Type":"ContainerStarted","Data":"dfd6bc2891ed37ffab84151f440b1a56ed6456a32e21850dc75ed9c5410befa7"}
Sep 10 21:16:03.091484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:03.091440    2573 generic.go:356] "Generic (PLEG): container finished" podID="d4a94661-aef3-4bc5-93f1-912309ffee46" containerID="37ed270e2e530ebe9fe32279dd7f70219385dba1cccc8189f5098ec2db825531" exitCode=0
Sep 10 21:16:03.091484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:03.091481    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22" event={"ID":"d4a94661-aef3-4bc5-93f1-912309ffee46","Type":"ContainerDied","Data":"37ed270e2e530ebe9fe32279dd7f70219385dba1cccc8189f5098ec2db825531"}
Sep 10 21:16:04.901603 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:04.901577    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vj6k8"]
Sep 10 21:16:04.915319 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:04.915292    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vj6k8"]
Sep 10 21:16:04.915454 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:04.915376    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8"
Sep 10 21:16:04.917450 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:04.917421    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-v9nr8\""
Sep 10 21:16:04.921177 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:04.921153    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:16:05.102278 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.102251    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22"
Sep 10 21:16:05.102430 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.102251    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdkq22" event={"ID":"d4a94661-aef3-4bc5-93f1-912309ffee46","Type":"ContainerDied","Data":"50b8ae8baca04f6057e3cefd789a1ae380670fa050009a4496cff7ba59ad2576"}
Sep 10 21:16:05.102430 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.102374    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50b8ae8baca04f6057e3cefd789a1ae380670fa050009a4496cff7ba59ad2576"
Sep 10 21:16:05.106510 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.106487    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-bundle\") pod \"d4a94661-aef3-4bc5-93f1-912309ffee46\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") "
Sep 10 21:16:05.106510 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.106521    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-util\") pod \"d4a94661-aef3-4bc5-93f1-912309ffee46\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") "
Sep 10 21:16:05.106697 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.106560    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp6wx\" (UniqueName: \"kubernetes.io/projected/d4a94661-aef3-4bc5-93f1-912309ffee46-kube-api-access-wp6wx\") pod \"d4a94661-aef3-4bc5-93f1-912309ffee46\" (UID: \"d4a94661-aef3-4bc5-93f1-912309ffee46\") "
Sep 10 21:16:05.106815 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.106789    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c16908ce-3987-4e92-8c68-f80bcc909967-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vj6k8\" (UID: \"c16908ce-3987-4e92-8c68-f80bcc909967\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8"
Sep 10 21:16:05.106953 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.106848    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk68c\" (UniqueName: \"kubernetes.io/projected/c16908ce-3987-4e92-8c68-f80bcc909967-kube-api-access-zk68c\") pod \"cert-manager-webhook-6486ffd997-vj6k8\" (UID: \"c16908ce-3987-4e92-8c68-f80bcc909967\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8"
Sep 10 21:16:05.106953 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.106930    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-bundle" (OuterVolumeSpecName: "bundle") pod "d4a94661-aef3-4bc5-93f1-912309ffee46" (UID: "d4a94661-aef3-4bc5-93f1-912309ffee46"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:16:05.108737 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.108710    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a94661-aef3-4bc5-93f1-912309ffee46-kube-api-access-wp6wx" (OuterVolumeSpecName: "kube-api-access-wp6wx") pod "d4a94661-aef3-4bc5-93f1-912309ffee46" (UID: "d4a94661-aef3-4bc5-93f1-912309ffee46"). InnerVolumeSpecName "kube-api-access-wp6wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:05.110984 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.110960    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-util" (OuterVolumeSpecName: "util") pod "d4a94661-aef3-4bc5-93f1-912309ffee46" (UID: "d4a94661-aef3-4bc5-93f1-912309ffee46"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:16:05.208070 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.208043    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c16908ce-3987-4e92-8c68-f80bcc909967-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vj6k8\" (UID: \"c16908ce-3987-4e92-8c68-f80bcc909967\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8"
Sep 10 21:16:05.208190 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.208084    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk68c\" (UniqueName: \"kubernetes.io/projected/c16908ce-3987-4e92-8c68-f80bcc909967-kube-api-access-zk68c\") pod \"cert-manager-webhook-6486ffd997-vj6k8\" (UID: \"c16908ce-3987-4e92-8c68-f80bcc909967\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8"
Sep 10 21:16:05.208190 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.208132    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-bundle\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:16:05.208190 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.208140    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a94661-aef3-4bc5-93f1-912309ffee46-util\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:16:05.208190 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.208149    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp6wx\" (UniqueName: \"kubernetes.io/projected/d4a94661-aef3-4bc5-93f1-912309ffee46-kube-api-access-wp6wx\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:16:05.220665 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.217748    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c16908ce-3987-4e92-8c68-f80bcc909967-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vj6k8\" (UID: \"c16908ce-3987-4e92-8c68-f80bcc909967\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8"
Sep 10 21:16:05.220665 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.217908    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk68c\" (UniqueName: \"kubernetes.io/projected/c16908ce-3987-4e92-8c68-f80bcc909967-kube-api-access-zk68c\") pod \"cert-manager-webhook-6486ffd997-vj6k8\" (UID: \"c16908ce-3987-4e92-8c68-f80bcc909967\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8"
Sep 10 21:16:05.224787 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.224747    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8"
Sep 10 21:16:05.499031 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:05.499002    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vj6k8"]
Sep 10 21:16:05.500765 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:16:05.500733    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc16908ce_3987_4e92_8c68_f80bcc909967.slice/crio-5170a05c8336161c5a3d640cc6127d5defb72e1d27aeb6794f999bf5786f11bd WatchSource:0}: Error finding container 5170a05c8336161c5a3d640cc6127d5defb72e1d27aeb6794f999bf5786f11bd: Status 404 returned error can't find the container with id 5170a05c8336161c5a3d640cc6127d5defb72e1d27aeb6794f999bf5786f11bd
Sep 10 21:16:06.108648 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:06.108610    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss" event={"ID":"6ae7e5c3-2b67-47b0-9506-63f06f0993a1","Type":"ContainerStarted","Data":"5573e15b36d8b8b2563117c2d451444a2974e176ce06fb9590fd2487a861afbb"}
Sep 10 21:16:06.110248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:06.110224    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8" event={"ID":"c16908ce-3987-4e92-8c68-f80bcc909967","Type":"ContainerStarted","Data":"d90dd958df8469b5e1006822a53d0c56159036b04a14d1863c08e7f4c31e0255"}
Sep 10 21:16:06.110374 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:06.110257    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8" event={"ID":"c16908ce-3987-4e92-8c68-f80bcc909967","Type":"ContainerStarted","Data":"5170a05c8336161c5a3d640cc6127d5defb72e1d27aeb6794f999bf5786f11bd"}
Sep 10 21:16:06.110374 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:06.110342    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8"
Sep 10 21:16:06.125561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:06.125511    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zn6ss" podStartSLOduration=2.477779863 podStartE2EDuration="5.125494863s" totalImagesPullingTime="2.647715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:06.124050729 +0000 UTC m=+342.057717549" watchObservedRunningTime="2026-09-10 21:16:06.125494863 +0000 UTC m=+342.059161677"
Sep 10 21:16:06.140062 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:06.140002    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8" podStartSLOduration=2.139980722 podStartE2EDuration="2.139980722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:06.137715128 +0000 UTC m=+342.071381944" watchObservedRunningTime="2026-09-10 21:16:06.139980722 +0000 UTC m=+342.073647538"
Sep 10 21:16:12.117551 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:12.117515    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-vj6k8"
Sep 10 21:16:18.237174 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.237094    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"]
Sep 10 21:16:18.237544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.237439    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a94661-aef3-4bc5-93f1-912309ffee46" containerName="util"
Sep 10 21:16:18.237544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.237454    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a94661-aef3-4bc5-93f1-912309ffee46" containerName="util"
Sep 10 21:16:18.237544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.237468    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a94661-aef3-4bc5-93f1-912309ffee46" containerName="pull"
Sep 10 21:16:18.237544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.237474    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a94661-aef3-4bc5-93f1-912309ffee46" containerName="pull"
Sep 10 21:16:18.237544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.237487    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a94661-aef3-4bc5-93f1-912309ffee46" containerName="extract"
Sep 10 21:16:18.237544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.237494    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a94661-aef3-4bc5-93f1-912309ffee46" containerName="extract"
Sep 10 21:16:18.237544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.237545    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a94661-aef3-4bc5-93f1-912309ffee46" containerName="extract"
Sep 10 21:16:18.258492 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.258461    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"]
Sep 10 21:16:18.258648 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.258578    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.261368 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.261343    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 21:16:18.261983 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.261957    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 21:16:18.262102 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.262030    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rpb92\""
Sep 10 21:16:18.297588 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.297547    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdrtt\" (UniqueName: \"kubernetes.io/projected/80171175-f6c9-436e-b091-9a80218513de-kube-api-access-xdrtt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.297588 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.297592    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.297847 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.297664    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.398889 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.398847    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.399080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.398911    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdrtt\" (UniqueName: \"kubernetes.io/projected/80171175-f6c9-436e-b091-9a80218513de-kube-api-access-xdrtt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.399080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.398942    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.399239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.399220    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.399276 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.399240    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.406349 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.406320    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdrtt\" (UniqueName: \"kubernetes.io/projected/80171175-f6c9-436e-b091-9a80218513de-kube-api-access-xdrtt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.568469 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.568386    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:18.856244 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:18.856221    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"]
Sep 10 21:16:18.858466 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:16:18.858439    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80171175_f6c9_436e_b091_9a80218513de.slice/crio-c13f8350398ca2e165c26e5691cc427d5a1be9cb6739a65e91fd66496140c4a5 WatchSource:0}: Error finding container c13f8350398ca2e165c26e5691cc427d5a1be9cb6739a65e91fd66496140c4a5: Status 404 returned error can't find the container with id c13f8350398ca2e165c26e5691cc427d5a1be9cb6739a65e91fd66496140c4a5
Sep 10 21:16:19.167715 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:19.167624    2573 generic.go:356] "Generic (PLEG): container finished" podID="80171175-f6c9-436e-b091-9a80218513de" containerID="8100a252494c37252db66a43ab32975bf6e98af947b4114fd577dfb75723c979" exitCode=0
Sep 10 21:16:19.167715 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:19.167703    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt" event={"ID":"80171175-f6c9-436e-b091-9a80218513de","Type":"ContainerDied","Data":"8100a252494c37252db66a43ab32975bf6e98af947b4114fd577dfb75723c979"}
Sep 10 21:16:19.167927 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:19.167752    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt" event={"ID":"80171175-f6c9-436e-b091-9a80218513de","Type":"ContainerStarted","Data":"c13f8350398ca2e165c26e5691cc427d5a1be9cb6739a65e91fd66496140c4a5"}
Sep 10 21:16:20.173918 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:20.173885    2573 generic.go:356] "Generic (PLEG): container finished" podID="80171175-f6c9-436e-b091-9a80218513de" containerID="b9918e7c3a29995044dfaafc98c4699f49c82f40739bbe7840deeb550753a0a1" exitCode=0
Sep 10 21:16:20.174297 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:20.173951    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt" event={"ID":"80171175-f6c9-436e-b091-9a80218513de","Type":"ContainerDied","Data":"b9918e7c3a29995044dfaafc98c4699f49c82f40739bbe7840deeb550753a0a1"}
Sep 10 21:16:21.180029 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:21.179994    2573 generic.go:356] "Generic (PLEG): container finished" podID="80171175-f6c9-436e-b091-9a80218513de" containerID="c196cd991260dd2cfad385ff9d199dc680d41b3e115686585b1d25ef80d745d6" exitCode=0
Sep 10 21:16:21.180418 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:21.180081    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt" event={"ID":"80171175-f6c9-436e-b091-9a80218513de","Type":"ContainerDied","Data":"c196cd991260dd2cfad385ff9d199dc680d41b3e115686585b1d25ef80d745d6"}
Sep 10 21:16:22.314552 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:22.311561    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:22.331426 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:22.331382    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-util\") pod \"80171175-f6c9-436e-b091-9a80218513de\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") "
Sep 10 21:16:22.331628 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:22.331435    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-bundle\") pod \"80171175-f6c9-436e-b091-9a80218513de\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") "
Sep 10 21:16:22.331628 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:22.331482    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdrtt\" (UniqueName: \"kubernetes.io/projected/80171175-f6c9-436e-b091-9a80218513de-kube-api-access-xdrtt\") pod \"80171175-f6c9-436e-b091-9a80218513de\" (UID: \"80171175-f6c9-436e-b091-9a80218513de\") "
Sep 10 21:16:22.332711 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:22.332676    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-bundle" (OuterVolumeSpecName: "bundle") pod "80171175-f6c9-436e-b091-9a80218513de" (UID: "80171175-f6c9-436e-b091-9a80218513de"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:16:22.333903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:22.333873    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80171175-f6c9-436e-b091-9a80218513de-kube-api-access-xdrtt" (OuterVolumeSpecName: "kube-api-access-xdrtt") pod "80171175-f6c9-436e-b091-9a80218513de" (UID: "80171175-f6c9-436e-b091-9a80218513de"). InnerVolumeSpecName "kube-api-access-xdrtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:22.339692 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:22.339664    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-util" (OuterVolumeSpecName: "util") pod "80171175-f6c9-436e-b091-9a80218513de" (UID: "80171175-f6c9-436e-b091-9a80218513de"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:16:22.432293 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:22.432243    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-util\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:16:22.432549 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:22.432309    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80171175-f6c9-436e-b091-9a80218513de-bundle\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:16:22.432549 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:22.432323    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdrtt\" (UniqueName: \"kubernetes.io/projected/80171175-f6c9-436e-b091-9a80218513de-kube-api-access-xdrtt\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:16:23.194268 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:23.194239    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt"
Sep 10 21:16:23.194268 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:23.194258    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzgkt" event={"ID":"80171175-f6c9-436e-b091-9a80218513de","Type":"ContainerDied","Data":"c13f8350398ca2e165c26e5691cc427d5a1be9cb6739a65e91fd66496140c4a5"}
Sep 10 21:16:23.194474 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:23.194299    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c13f8350398ca2e165c26e5691cc427d5a1be9cb6739a65e91fd66496140c4a5"
Sep 10 21:16:34.266229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.266191    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"]
Sep 10 21:16:34.266734 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.266522    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80171175-f6c9-436e-b091-9a80218513de" containerName="pull"
Sep 10 21:16:34.266734 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.266534    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="80171175-f6c9-436e-b091-9a80218513de" containerName="pull"
Sep 10 21:16:34.266734 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.266543    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80171175-f6c9-436e-b091-9a80218513de" containerName="extract"
Sep 10 21:16:34.266734 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.266548    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="80171175-f6c9-436e-b091-9a80218513de" containerName="extract"
Sep 10 21:16:34.266734 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.266569    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80171175-f6c9-436e-b091-9a80218513de" containerName="util"
Sep 10 21:16:34.266734 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.266575    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="80171175-f6c9-436e-b091-9a80218513de" containerName="util"
Sep 10 21:16:34.266734 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.266630    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="80171175-f6c9-436e-b091-9a80218513de" containerName="extract"
Sep 10 21:16:34.269882 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.269861    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.272148 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.272121    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 21:16:34.272296 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.272165    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 21:16:34.272296 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.272208    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rpb92\""
Sep 10 21:16:34.278406 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.278379    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"]
Sep 10 21:16:34.331882 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.331840    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.332058 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.331898    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.332058 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.331925    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4xdl\" (UniqueName: \"kubernetes.io/projected/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-kube-api-access-b4xdl\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.433353 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.433294    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.433541 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.433352    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4xdl\" (UniqueName: \"kubernetes.io/projected/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-kube-api-access-b4xdl\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.433541 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.433472    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.433704 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.433683    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.433928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.433900    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.443877 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.443848    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4xdl\" (UniqueName: \"kubernetes.io/projected/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-kube-api-access-b4xdl\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.579881 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.579844    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:34.857602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:34.857574    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"]
Sep 10 21:16:34.859838 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:16:34.859801    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1da684_5ba9_46c5_aaf6_c6447e5aa987.slice/crio-1eb1d336c1e2ea0d7af0e6ce50e3375d3a47d46706d03e796c2f67546993035d WatchSource:0}: Error finding container 1eb1d336c1e2ea0d7af0e6ce50e3375d3a47d46706d03e796c2f67546993035d: Status 404 returned error can't find the container with id 1eb1d336c1e2ea0d7af0e6ce50e3375d3a47d46706d03e796c2f67546993035d
Sep 10 21:16:35.246093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.246003    2573 generic.go:356] "Generic (PLEG): container finished" podID="2b1da684-5ba9-46c5-aaf6-c6447e5aa987" containerID="5b4af7409dc970a960de639241e181ee98e0edb3892aff2f960e40f3e14503a4" exitCode=0
Sep 10 21:16:35.246246 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.246094    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj" event={"ID":"2b1da684-5ba9-46c5-aaf6-c6447e5aa987","Type":"ContainerDied","Data":"5b4af7409dc970a960de639241e181ee98e0edb3892aff2f960e40f3e14503a4"}
Sep 10 21:16:35.246246 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.246149    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj" event={"ID":"2b1da684-5ba9-46c5-aaf6-c6447e5aa987","Type":"ContainerStarted","Data":"1eb1d336c1e2ea0d7af0e6ce50e3375d3a47d46706d03e796c2f67546993035d"}
Sep 10 21:16:35.613219 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.613181    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"]
Sep 10 21:16:35.615917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.615893    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.618511 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.618489    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 21:16:35.618511 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.618499    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 21:16:35.618708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.618504    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fpp5b\""
Sep 10 21:16:35.618835 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.618816    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 21:16:35.618911 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.618858    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 21:16:35.634567 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.634537    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"]
Sep 10 21:16:35.644411 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.644379    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xr9p\" (UniqueName: \"kubernetes.io/projected/8db3bd51-745a-4ec9-9c39-4145c3a43dac-kube-api-access-4xr9p\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.644411 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.644422    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8db3bd51-745a-4ec9-9c39-4145c3a43dac-tmp\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.644618 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.644479    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8db3bd51-745a-4ec9-9c39-4145c3a43dac-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.644618 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.644519    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8db3bd51-745a-4ec9-9c39-4145c3a43dac-manifests\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.644618 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.644535    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8db3bd51-745a-4ec9-9c39-4145c3a43dac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.745374 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.745335    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8db3bd51-745a-4ec9-9c39-4145c3a43dac-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.745558 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.745386    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8db3bd51-745a-4ec9-9c39-4145c3a43dac-manifests\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.745558 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.745405    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8db3bd51-745a-4ec9-9c39-4145c3a43dac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.745558 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.745434    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xr9p\" (UniqueName: \"kubernetes.io/projected/8db3bd51-745a-4ec9-9c39-4145c3a43dac-kube-api-access-4xr9p\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.745558 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.745464    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8db3bd51-745a-4ec9-9c39-4145c3a43dac-tmp\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.745864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.745839    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8db3bd51-745a-4ec9-9c39-4145c3a43dac-manifests\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.747924 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.747892    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8db3bd51-745a-4ec9-9c39-4145c3a43dac-tmp\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.748047 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.747995    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8db3bd51-745a-4ec9-9c39-4145c3a43dac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.748126 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.748107    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8db3bd51-745a-4ec9-9c39-4145c3a43dac-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.754588 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.754561    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xr9p\" (UniqueName: \"kubernetes.io/projected/8db3bd51-745a-4ec9-9c39-4145c3a43dac-kube-api-access-4xr9p\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd\" (UID: \"8db3bd51-745a-4ec9-9c39-4145c3a43dac\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:35.926618 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:35.926529    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:36.243043 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:36.243017    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"]
Sep 10 21:16:36.244525 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:16:36.244501    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db3bd51_745a_4ec9_9c39_4145c3a43dac.slice/crio-89a0a698dba16177deb540b476ab13033452d7aed674e2952c4e63c64232aae6 WatchSource:0}: Error finding container 89a0a698dba16177deb540b476ab13033452d7aed674e2952c4e63c64232aae6: Status 404 returned error can't find the container with id 89a0a698dba16177deb540b476ab13033452d7aed674e2952c4e63c64232aae6
Sep 10 21:16:36.253165 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:36.253092    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj" event={"ID":"2b1da684-5ba9-46c5-aaf6-c6447e5aa987","Type":"ContainerStarted","Data":"b9aa9781b89c9d8cbef9dfbaa0544bdc79cec20c9688a8f6aa7872b895223011"}
Sep 10 21:16:36.254469 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:36.254446    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd" event={"ID":"8db3bd51-745a-4ec9-9c39-4145c3a43dac","Type":"ContainerStarted","Data":"89a0a698dba16177deb540b476ab13033452d7aed674e2952c4e63c64232aae6"}
Sep 10 21:16:37.267947 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:37.267912    2573 generic.go:356] "Generic (PLEG): container finished" podID="2b1da684-5ba9-46c5-aaf6-c6447e5aa987" containerID="b9aa9781b89c9d8cbef9dfbaa0544bdc79cec20c9688a8f6aa7872b895223011" exitCode=0
Sep 10 21:16:37.268418 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:37.267982    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj" event={"ID":"2b1da684-5ba9-46c5-aaf6-c6447e5aa987","Type":"ContainerDied","Data":"b9aa9781b89c9d8cbef9dfbaa0544bdc79cec20c9688a8f6aa7872b895223011"}
Sep 10 21:16:38.277048 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:38.277006    2573 generic.go:356] "Generic (PLEG): container finished" podID="2b1da684-5ba9-46c5-aaf6-c6447e5aa987" containerID="4435f1ac66e5905b27150cc83098a8ace24aba1a3735eb20040fd3b41ed51ca3" exitCode=0
Sep 10 21:16:38.277455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:38.277083    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj" event={"ID":"2b1da684-5ba9-46c5-aaf6-c6447e5aa987","Type":"ContainerDied","Data":"4435f1ac66e5905b27150cc83098a8ace24aba1a3735eb20040fd3b41ed51ca3"}
Sep 10 21:16:39.422846 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:39.422816    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:39.479506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:39.479474    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-bundle\") pod \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") "
Sep 10 21:16:39.479676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:39.479550    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-util\") pod \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") "
Sep 10 21:16:39.479676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:39.479626    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4xdl\" (UniqueName: \"kubernetes.io/projected/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-kube-api-access-b4xdl\") pod \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\" (UID: \"2b1da684-5ba9-46c5-aaf6-c6447e5aa987\") "
Sep 10 21:16:39.480344 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:39.480316    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-bundle" (OuterVolumeSpecName: "bundle") pod "2b1da684-5ba9-46c5-aaf6-c6447e5aa987" (UID: "2b1da684-5ba9-46c5-aaf6-c6447e5aa987"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:16:39.481739 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:39.481712    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-kube-api-access-b4xdl" (OuterVolumeSpecName: "kube-api-access-b4xdl") pod "2b1da684-5ba9-46c5-aaf6-c6447e5aa987" (UID: "2b1da684-5ba9-46c5-aaf6-c6447e5aa987"). InnerVolumeSpecName "kube-api-access-b4xdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:39.485268 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:39.485231    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-util" (OuterVolumeSpecName: "util") pod "2b1da684-5ba9-46c5-aaf6-c6447e5aa987" (UID: "2b1da684-5ba9-46c5-aaf6-c6447e5aa987"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:16:39.580289 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:39.580244    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4xdl\" (UniqueName: \"kubernetes.io/projected/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-kube-api-access-b4xdl\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:16:39.580289 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:39.580274    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-bundle\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:16:39.580289 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:39.580285    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b1da684-5ba9-46c5-aaf6-c6447e5aa987-util\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:16:40.288872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:40.288844    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj"
Sep 10 21:16:40.288872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:40.288860    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0j9gqj" event={"ID":"2b1da684-5ba9-46c5-aaf6-c6447e5aa987","Type":"ContainerDied","Data":"1eb1d336c1e2ea0d7af0e6ce50e3375d3a47d46706d03e796c2f67546993035d"}
Sep 10 21:16:40.289235 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:40.288908    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1eb1d336c1e2ea0d7af0e6ce50e3375d3a47d46706d03e796c2f67546993035d"
Sep 10 21:16:40.290513 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:40.290477    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd" event={"ID":"8db3bd51-745a-4ec9-9c39-4145c3a43dac","Type":"ContainerStarted","Data":"acbe3ba4daecbee68315ecba1a58f76b5050bc18640c6bfc4dcac6748824645c"}
Sep 10 21:16:41.297243 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:41.297206    2573 generic.go:356] "Generic (PLEG): container finished" podID="8db3bd51-745a-4ec9-9c39-4145c3a43dac" containerID="acbe3ba4daecbee68315ecba1a58f76b5050bc18640c6bfc4dcac6748824645c" exitCode=0
Sep 10 21:16:41.297626 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:41.297286    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd" event={"ID":"8db3bd51-745a-4ec9-9c39-4145c3a43dac","Type":"ContainerDied","Data":"acbe3ba4daecbee68315ecba1a58f76b5050bc18640c6bfc4dcac6748824645c"}
Sep 10 21:16:42.304795 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:42.304743    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd" event={"ID":"8db3bd51-745a-4ec9-9c39-4145c3a43dac","Type":"ContainerStarted","Data":"b9be9e8f30c5af780b22dc32e0e94d9584fa4d354079779bc02bdac8cb7dffff"}
Sep 10 21:16:42.305179 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:42.304836    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:42.332948 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:42.332891    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd" podStartSLOduration=2.781342019 podStartE2EDuration="7.332874417s" totalImagesPullingTime="3.474269039s" totalInitContainerRuntime="1.077263359s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:42.330331482 +0000 UTC m=+378.263998298" watchObservedRunningTime="2026-09-10 21:16:42.332874417 +0000 UTC m=+378.266541231"
Sep 10 21:16:53.312432 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:53.312400    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd"
Sep 10 21:16:55.309234 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.309199    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"]
Sep 10 21:16:55.309641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.309535    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b1da684-5ba9-46c5-aaf6-c6447e5aa987" containerName="pull"
Sep 10 21:16:55.309641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.309548    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b1da684-5ba9-46c5-aaf6-c6447e5aa987" containerName="pull"
Sep 10 21:16:55.309641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.309571    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b1da684-5ba9-46c5-aaf6-c6447e5aa987" containerName="util"
Sep 10 21:16:55.309641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.309577    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b1da684-5ba9-46c5-aaf6-c6447e5aa987" containerName="util"
Sep 10 21:16:55.309641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.309586    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b1da684-5ba9-46c5-aaf6-c6447e5aa987" containerName="extract"
Sep 10 21:16:55.309641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.309592    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b1da684-5ba9-46c5-aaf6-c6447e5aa987" containerName="extract"
Sep 10 21:16:55.309867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.309648    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b1da684-5ba9-46c5-aaf6-c6447e5aa987" containerName="extract"
Sep 10 21:16:55.312104 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.312088    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.315828 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.315802    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 21:16:55.316024 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.316004    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bxnvc\""
Sep 10 21:16:55.316465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.316451    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 21:16:55.317043 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.317018    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 21:16:55.317150 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.317135    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 21:16:55.331251 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.331219    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"]
Sep 10 21:16:55.419888 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.419854    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d-tls-certs\") pod \"kube-auth-proxy-8545c498cb-gc9s9\" (UID: \"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.419888 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.419893    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fffct\" (UniqueName: \"kubernetes.io/projected/79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d-kube-api-access-fffct\") pod \"kube-auth-proxy-8545c498cb-gc9s9\" (UID: \"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.420126 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.419921    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d-tmp\") pod \"kube-auth-proxy-8545c498cb-gc9s9\" (UID: \"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.521358 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.521319    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d-tls-certs\") pod \"kube-auth-proxy-8545c498cb-gc9s9\" (UID: \"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.521358 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.521362    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fffct\" (UniqueName: \"kubernetes.io/projected/79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d-kube-api-access-fffct\") pod \"kube-auth-proxy-8545c498cb-gc9s9\" (UID: \"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.521609 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.521386    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d-tmp\") pod \"kube-auth-proxy-8545c498cb-gc9s9\" (UID: \"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.523687 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.523658    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d-tmp\") pod \"kube-auth-proxy-8545c498cb-gc9s9\" (UID: \"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.523966 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.523947    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d-tls-certs\") pod \"kube-auth-proxy-8545c498cb-gc9s9\" (UID: \"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.532287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.532262    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fffct\" (UniqueName: \"kubernetes.io/projected/79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d-kube-api-access-fffct\") pod \"kube-auth-proxy-8545c498cb-gc9s9\" (UID: \"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.622004 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.621959    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"
Sep 10 21:16:55.897044 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:55.897018    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9"]
Sep 10 21:16:55.899118 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:16:55.899089    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ac50cb_b3d4_4c21_a0c3_80dc4b58ae4d.slice/crio-19c306a9cf3d5c6e00cced852452c799d061295c06aaa443531f7efb02a0e998 WatchSource:0}: Error finding container 19c306a9cf3d5c6e00cced852452c799d061295c06aaa443531f7efb02a0e998: Status 404 returned error can't find the container with id 19c306a9cf3d5c6e00cced852452c799d061295c06aaa443531f7efb02a0e998
Sep 10 21:16:56.372820 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:56.372769    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9" event={"ID":"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d","Type":"ContainerStarted","Data":"19c306a9cf3d5c6e00cced852452c799d061295c06aaa443531f7efb02a0e998"}
Sep 10 21:16:56.873092 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:56.873049    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"]
Sep 10 21:16:56.876308 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:56.876276    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:56.878631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:56.878364    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 21:16:56.878631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:56.878424    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 21:16:56.878631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:56.878478    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 21:16:56.878631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:56.878615    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 21:16:56.878631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:56.878630    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nxgtr\""
Sep 10 21:16:56.892145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:56.892111    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"]
Sep 10 21:16:57.035766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.035608    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.035766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.035683    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfrbv\" (UniqueName: \"kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-kube-api-access-wfrbv\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.035766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.035732    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.036098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.035867    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61a83d4-c3fa-4f53-9661-dec7f50f5339-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.036098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.035890    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.036098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.035914    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.036098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.035952    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.137031 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.136866    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.137031 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.136944    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.137261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.137089    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfrbv\" (UniqueName: \"kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-kube-api-access-wfrbv\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.137261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.137174    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.137261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.137233    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61a83d4-c3fa-4f53-9661-dec7f50f5339-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.137261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.137256    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.137474 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.137285    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.137612 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.137585    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.140162 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.140108    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.140284 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.140229    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.140518 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.140496    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61a83d4-c3fa-4f53-9661-dec7f50f5339-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.141367 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.141340    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.145345 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.145312    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.145617 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.145590    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfrbv\" (UniqueName: \"kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-kube-api-access-wfrbv\") pod \"istiod-openshift-gateway-7b7995d5cd-dzfxg\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:57.191124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:57.191083    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:16:58.792546 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:58.792481    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"]
Sep 10 21:16:58.794725 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:16:58.794684    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda61a83d4_c3fa_4f53_9661_dec7f50f5339.slice/crio-fbf01154940d261a5b24fa21a184fc8fc618aac170ecd178ec260609bc97b135 WatchSource:0}: Error finding container fbf01154940d261a5b24fa21a184fc8fc618aac170ecd178ec260609bc97b135: Status 404 returned error can't find the container with id fbf01154940d261a5b24fa21a184fc8fc618aac170ecd178ec260609bc97b135
Sep 10 21:16:59.393179 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:59.393135    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg" event={"ID":"a61a83d4-c3fa-4f53-9661-dec7f50f5339","Type":"ContainerStarted","Data":"fbf01154940d261a5b24fa21a184fc8fc618aac170ecd178ec260609bc97b135"}
Sep 10 21:16:59.394871 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:59.394838    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9" event={"ID":"79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d","Type":"ContainerStarted","Data":"b2617905b5d2201e12323b374b1ccd83db7e231d5fbbae357d9360d7936c0206"}
Sep 10 21:16:59.411436 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:16:59.411318    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8545c498cb-gc9s9" podStartSLOduration=1.7519976499999999 podStartE2EDuration="4.41129633s" totalImagesPullingTime="2.65929868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:59.410243882 +0000 UTC m=+395.343910698" watchObservedRunningTime="2026-09-10 21:16:59.41129633 +0000 UTC m=+395.344963146"
Sep 10 21:17:01.216981 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:01.216943    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:17:01.217259 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:01.217024    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:17:01.406465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:01.406431    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg" event={"ID":"a61a83d4-c3fa-4f53-9661-dec7f50f5339","Type":"ContainerStarted","Data":"5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a"}
Sep 10 21:17:01.406635 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:01.406507    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:17:01.428505 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:01.428449    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg" podStartSLOduration=3.008772571 podStartE2EDuration="5.428431048s" totalImagesPullingTime="2.419658477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:01.427351346 +0000 UTC m=+397.361018186" watchObservedRunningTime="2026-09-10 21:17:01.428431048 +0000 UTC m=+397.362097862"
Sep 10 21:17:02.415212 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:02.415171    2573 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-dzfxg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:17:02.415691 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:02.415252    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg" podUID="a61a83d4-c3fa-4f53-9661-dec7f50f5339" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:17:05.413996 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:05.413957    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:17:13.933557 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.933510    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"]
Sep 10 21:17:13.955544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.955508    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"]
Sep 10 21:17:13.955704 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.955667    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:13.957826 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.957801    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:17:13.958569 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.958544    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 21:17:13.958705 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.958641    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c9hw7\""
Sep 10 21:17:13.958705 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.958688    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 21:17:13.958999 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.958955    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 21:17:13.958999 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.958989    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 21:17:13.985950 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.985905    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k22s7\" (UniqueName: \"kubernetes.io/projected/847954c3-5929-4ea4-a393-31270ca58fb2-kube-api-access-k22s7\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:13.986160 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.986040    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/847954c3-5929-4ea4-a393-31270ca58fb2-metrics-cert\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:13.986160 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.986069    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/847954c3-5929-4ea4-a393-31270ca58fb2-cert\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:13.986160 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:13.986101    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/847954c3-5929-4ea4-a393-31270ca58fb2-manager-config\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:14.087017 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:14.086970    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/847954c3-5929-4ea4-a393-31270ca58fb2-manager-config\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:14.087220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:14.087050    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k22s7\" (UniqueName: \"kubernetes.io/projected/847954c3-5929-4ea4-a393-31270ca58fb2-kube-api-access-k22s7\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:14.087220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:14.087145    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/847954c3-5929-4ea4-a393-31270ca58fb2-metrics-cert\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:14.087220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:14.087166    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/847954c3-5929-4ea4-a393-31270ca58fb2-cert\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:14.087903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:14.087877    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/847954c3-5929-4ea4-a393-31270ca58fb2-manager-config\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:14.089957 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:14.089911    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/847954c3-5929-4ea4-a393-31270ca58fb2-cert\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:14.090068 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:14.089972    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/847954c3-5929-4ea4-a393-31270ca58fb2-metrics-cert\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:14.099062 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:14.099027    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k22s7\" (UniqueName: \"kubernetes.io/projected/847954c3-5929-4ea4-a393-31270ca58fb2-kube-api-access-k22s7\") pod \"lws-controller-manager-66f6f49f76-n7hvf\" (UID: \"847954c3-5929-4ea4-a393-31270ca58fb2\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:14.266873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:14.266762    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:14.564457 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:14.564432    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"]
Sep 10 21:17:14.566687 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:17:14.566649    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847954c3_5929_4ea4_a393_31270ca58fb2.slice/crio-3613db71e87b9b8f81274e1eee8a286a1b87e65784aced2d9ef98b68069c9f28 WatchSource:0}: Error finding container 3613db71e87b9b8f81274e1eee8a286a1b87e65784aced2d9ef98b68069c9f28: Status 404 returned error can't find the container with id 3613db71e87b9b8f81274e1eee8a286a1b87e65784aced2d9ef98b68069c9f28
Sep 10 21:17:15.476265 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:15.476230    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf" event={"ID":"847954c3-5929-4ea4-a393-31270ca58fb2","Type":"ContainerStarted","Data":"3613db71e87b9b8f81274e1eee8a286a1b87e65784aced2d9ef98b68069c9f28"}
Sep 10 21:17:16.485053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:16.485011    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf" event={"ID":"847954c3-5929-4ea4-a393-31270ca58fb2","Type":"ContainerStarted","Data":"cb3f84c8b5d4acfb0eae6900a977273319b8d065b0d7f99328496fac719a8ec5"}
Sep 10 21:17:16.485531 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:16.485175    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:17:16.507004 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:16.506938    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf" podStartSLOduration=2.027693282 podStartE2EDuration="3.506918259s" totalImagesPullingTime="1.479224977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:16.50503772 +0000 UTC m=+412.438704535" watchObservedRunningTime="2026-09-10 21:17:16.506918259 +0000 UTC m=+412.440585074"
Sep 10 21:17:20.772773 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:20.772733    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"]
Sep 10 21:17:20.773247 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:20.772944    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg" podUID="a61a83d4-c3fa-4f53-9661-dec7f50f5339" containerName="discovery" containerID="cri-o://5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a" gracePeriod=30
Sep 10 21:17:21.017452 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.017421    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:17:21.055707 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.055616    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61a83d4-c3fa-4f53-9661-dec7f50f5339-local-certs\") pod \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") "
Sep 10 21:17:21.055707 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.055676    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfrbv\" (UniqueName: \"kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-kube-api-access-wfrbv\") pod \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") "
Sep 10 21:17:21.055707 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.055697    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-dns-cert\") pod \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") "
Sep 10 21:17:21.056008 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.055733    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-ca-configmap\") pod \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") "
Sep 10 21:17:21.056061 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.056022    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-token\") pod \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") "
Sep 10 21:17:21.056146 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.056118    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-kubeconfig\") pod \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") "
Sep 10 21:17:21.056242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.056181    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-cacerts\") pod \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\" (UID: \"a61a83d4-c3fa-4f53-9661-dec7f50f5339\") "
Sep 10 21:17:21.056563 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.056218    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "a61a83d4-c3fa-4f53-9661-dec7f50f5339" (UID: "a61a83d4-c3fa-4f53-9661-dec7f50f5339"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:17:21.058351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.058316    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "a61a83d4-c3fa-4f53-9661-dec7f50f5339" (UID: "a61a83d4-c3fa-4f53-9661-dec7f50f5339"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:17:21.058481 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.058422    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61a83d4-c3fa-4f53-9661-dec7f50f5339-local-certs" (OuterVolumeSpecName: "local-certs") pod "a61a83d4-c3fa-4f53-9661-dec7f50f5339" (UID: "a61a83d4-c3fa-4f53-9661-dec7f50f5339"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:17:21.058960 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.058928    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-cacerts" (OuterVolumeSpecName: "cacerts") pod "a61a83d4-c3fa-4f53-9661-dec7f50f5339" (UID: "a61a83d4-c3fa-4f53-9661-dec7f50f5339"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:17:21.059045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.058963    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "a61a83d4-c3fa-4f53-9661-dec7f50f5339" (UID: "a61a83d4-c3fa-4f53-9661-dec7f50f5339"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:17:21.059045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.059006    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-kube-api-access-wfrbv" (OuterVolumeSpecName: "kube-api-access-wfrbv") pod "a61a83d4-c3fa-4f53-9661-dec7f50f5339" (UID: "a61a83d4-c3fa-4f53-9661-dec7f50f5339"). InnerVolumeSpecName "kube-api-access-wfrbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:17:21.059045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.059006    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-token" (OuterVolumeSpecName: "istio-token") pod "a61a83d4-c3fa-4f53-9661-dec7f50f5339" (UID: "a61a83d4-c3fa-4f53-9661-dec7f50f5339"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:17:21.157627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.157572    2573 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-cacerts\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:17:21.157627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.157616    2573 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61a83d4-c3fa-4f53-9661-dec7f50f5339-local-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:17:21.157627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.157628    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfrbv\" (UniqueName: \"kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-kube-api-access-wfrbv\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:17:21.157627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.157637    2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-dns-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:17:21.157627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.157648    2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-csr-ca-configmap\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:17:21.157985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.157658    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:17:21.157985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.157666    2573 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a61a83d4-c3fa-4f53-9661-dec7f50f5339-istio-kubeconfig\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:17:21.509993 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.509955    2573 generic.go:356] "Generic (PLEG): container finished" podID="a61a83d4-c3fa-4f53-9661-dec7f50f5339" containerID="5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a" exitCode=0
Sep 10 21:17:21.510202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.510004    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"
Sep 10 21:17:21.510202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.510010    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg" event={"ID":"a61a83d4-c3fa-4f53-9661-dec7f50f5339","Type":"ContainerDied","Data":"5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a"}
Sep 10 21:17:21.510202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.510064    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg" event={"ID":"a61a83d4-c3fa-4f53-9661-dec7f50f5339","Type":"ContainerDied","Data":"fbf01154940d261a5b24fa21a184fc8fc618aac170ecd178ec260609bc97b135"}
Sep 10 21:17:21.510202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.510090    2573 scope.go:117] "RemoveContainer" containerID="5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a"
Sep 10 21:17:21.519708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.519684    2573 scope.go:117] "RemoveContainer" containerID="5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a"
Sep 10 21:17:21.520099 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:17:21.520078    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a\": container with ID starting with 5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a not found: ID does not exist" containerID="5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a"
Sep 10 21:17:21.520165 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.520124    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a"} err="failed to get container status \"5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a\": rpc error: code = NotFound desc = could not find container \"5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a\": container with ID starting with 5e58ee9b1c0799ed3ac5b33ca797b69655307b9b300b24a16d1c2ee73297e45a not found: ID does not exist"
Sep 10 21:17:21.538434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.538391    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"]
Sep 10 21:17:21.547481 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:21.547446    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dzfxg"]
Sep 10 21:17:22.679001 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:22.678969    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a61a83d4-c3fa-4f53-9661-dec7f50f5339" path="/var/lib/kubelet/pods/a61a83d4-c3fa-4f53-9661-dec7f50f5339/volumes"
Sep 10 21:17:27.492576 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:17:27.492525    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-n7hvf"
Sep 10 21:18:10.736974 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.736933    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"]
Sep 10 21:18:10.737517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.737421    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a61a83d4-c3fa-4f53-9661-dec7f50f5339" containerName="discovery"
Sep 10 21:18:10.737517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.737442    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a61a83d4-c3fa-4f53-9661-dec7f50f5339" containerName="discovery"
Sep 10 21:18:10.737640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.737531    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a61a83d4-c3fa-4f53-9661-dec7f50f5339" containerName="discovery"
Sep 10 21:18:10.739972 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.739949    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:10.741926 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.741895    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rpb92\""
Sep 10 21:18:10.742147 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.742132    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 21:18:10.742442 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.742412    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 21:18:10.748205 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.748180    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"]
Sep 10 21:18:10.803450 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.803403    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjmm7\" (UniqueName: \"kubernetes.io/projected/c2d88a40-623e-4128-a05b-93cd7499a5bb-kube-api-access-fjmm7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:10.803678 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.803491    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:10.803678 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.803512    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:10.833446 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.833409    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"]
Sep 10 21:18:10.836026 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.836000    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:10.844568 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.844542    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"]
Sep 10 21:18:10.904355 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.904318    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg6qq\" (UniqueName: \"kubernetes.io/projected/3c7c216e-033d-4708-81bb-ffeac2ece578-kube-api-access-kg6qq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:10.904548 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.904381    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:10.904548 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.904408    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:10.904548 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.904427    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:10.904548 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.904494    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjmm7\" (UniqueName: \"kubernetes.io/projected/c2d88a40-623e-4128-a05b-93cd7499a5bb-kube-api-access-fjmm7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:10.904548 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.904539    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:10.904918 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.904848    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:10.904918 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.904894    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:10.912430 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.912405    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjmm7\" (UniqueName: \"kubernetes.io/projected/c2d88a40-623e-4128-a05b-93cd7499a5bb-kube-api-access-fjmm7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:10.935515 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.935479    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"]
Sep 10 21:18:10.938104 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.938088    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:10.947119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:10.947091    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"]
Sep 10 21:18:11.005478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.005385    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lkfp\" (UniqueName: \"kubernetes.io/projected/c7dd40f6-d94d-4857-a8ad-630c4406af18-kube-api-access-5lkfp\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:11.005478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.005434    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:11.005478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.005461    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg6qq\" (UniqueName: \"kubernetes.io/projected/3c7c216e-033d-4708-81bb-ffeac2ece578-kube-api-access-kg6qq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:11.005808 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.005497    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:11.005808 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.005520    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:11.005808 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.005565    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:11.005990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.005819    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:11.005990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.005879    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:11.016670 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.016640    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg6qq\" (UniqueName: \"kubernetes.io/projected/3c7c216e-033d-4708-81bb-ffeac2ece578-kube-api-access-kg6qq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:11.035347 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.035310    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"]
Sep 10 21:18:11.038070 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.038051    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.046659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.046631    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"]
Sep 10 21:18:11.050340 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.050318    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:11.106443 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.106412    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:11.106565 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.106478    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:11.106565 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.106532    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lkfp\" (UniqueName: \"kubernetes.io/projected/c7dd40f6-d94d-4857-a8ad-630c4406af18-kube-api-access-5lkfp\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:11.106689 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.106579    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.106689 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.106612    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.106689 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.106642    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94tlw\" (UniqueName: \"kubernetes.io/projected/a8c2eab6-edeb-497b-9674-bae502e2e029-kube-api-access-94tlw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.106957 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.106924    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:11.107024 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.107003    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:11.117271 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.117237    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lkfp\" (UniqueName: \"kubernetes.io/projected/c7dd40f6-d94d-4857-a8ad-630c4406af18-kube-api-access-5lkfp\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:11.145700 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.145668    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:11.207390 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.207352    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.207390 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.207392    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.207654 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.207414    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94tlw\" (UniqueName: \"kubernetes.io/projected/a8c2eab6-edeb-497b-9674-bae502e2e029-kube-api-access-94tlw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.207891 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.207853    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.207981 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.207906    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.215930 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.215896    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94tlw\" (UniqueName: \"kubernetes.io/projected/a8c2eab6-edeb-497b-9674-bae502e2e029-kube-api-access-94tlw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.248229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.248191    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:11.322951 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.322923    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"]
Sep 10 21:18:11.325116 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:18:11.325089    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2d88a40_623e_4128_a05b_93cd7499a5bb.slice/crio-5e9f904504d3b748e89d642586c19f4ff3c5a4cae35d0c5253d16b408dbe68cd WatchSource:0}: Error finding container 5e9f904504d3b748e89d642586c19f4ff3c5a4cae35d0c5253d16b408dbe68cd: Status 404 returned error can't find the container with id 5e9f904504d3b748e89d642586c19f4ff3c5a4cae35d0c5253d16b408dbe68cd
Sep 10 21:18:11.349094 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.349068    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:11.416839 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.416742    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"]
Sep 10 21:18:11.418528 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:18:11.418499    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c7c216e_033d_4708_81bb_ffeac2ece578.slice/crio-2d57d67d6ebb18db8a07d9d30d2e58455e5c904392ff335ad1f12d96b06410ec WatchSource:0}: Error finding container 2d57d67d6ebb18db8a07d9d30d2e58455e5c904392ff335ad1f12d96b06410ec: Status 404 returned error can't find the container with id 2d57d67d6ebb18db8a07d9d30d2e58455e5c904392ff335ad1f12d96b06410ec
Sep 10 21:18:11.517252 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.517228    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"]
Sep 10 21:18:11.519500 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:18:11.519477    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7dd40f6_d94d_4857_a8ad_630c4406af18.slice/crio-a76544b6b89bfaba4206b7c4e90aca01217bee15dad3d10e5e1d91d71da75f10 WatchSource:0}: Error finding container a76544b6b89bfaba4206b7c4e90aca01217bee15dad3d10e5e1d91d71da75f10: Status 404 returned error can't find the container with id a76544b6b89bfaba4206b7c4e90aca01217bee15dad3d10e5e1d91d71da75f10
Sep 10 21:18:11.622164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.622134    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"]
Sep 10 21:18:11.623640 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:18:11.623614    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c2eab6_edeb_497b_9674_bae502e2e029.slice/crio-2371bd9e1a6b392236e5fdfd1b12342031d0bc949f71b3a8ed50a69685677158 WatchSource:0}: Error finding container 2371bd9e1a6b392236e5fdfd1b12342031d0bc949f71b3a8ed50a69685677158: Status 404 returned error can't find the container with id 2371bd9e1a6b392236e5fdfd1b12342031d0bc949f71b3a8ed50a69685677158
Sep 10 21:18:11.752003 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.751969    2573 generic.go:356] "Generic (PLEG): container finished" podID="c7dd40f6-d94d-4857-a8ad-630c4406af18" containerID="2e77a84c2bdd2acbd9297c55b3efe02f94214b1f68ce1a53fe1998b7f77fb6a9" exitCode=0
Sep 10 21:18:11.752454 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.752063    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44" event={"ID":"c7dd40f6-d94d-4857-a8ad-630c4406af18","Type":"ContainerDied","Data":"2e77a84c2bdd2acbd9297c55b3efe02f94214b1f68ce1a53fe1998b7f77fb6a9"}
Sep 10 21:18:11.752454 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.752125    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44" event={"ID":"c7dd40f6-d94d-4857-a8ad-630c4406af18","Type":"ContainerStarted","Data":"a76544b6b89bfaba4206b7c4e90aca01217bee15dad3d10e5e1d91d71da75f10"}
Sep 10 21:18:11.753753 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.753729    2573 generic.go:356] "Generic (PLEG): container finished" podID="c2d88a40-623e-4128-a05b-93cd7499a5bb" containerID="897fc7bb76a0b2e7e42b4071f6a1df3dfd4e24f5adf735d44766f74e9e4549c5" exitCode=0
Sep 10 21:18:11.753864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.753814    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz" event={"ID":"c2d88a40-623e-4128-a05b-93cd7499a5bb","Type":"ContainerDied","Data":"897fc7bb76a0b2e7e42b4071f6a1df3dfd4e24f5adf735d44766f74e9e4549c5"}
Sep 10 21:18:11.753917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.753863    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz" event={"ID":"c2d88a40-623e-4128-a05b-93cd7499a5bb","Type":"ContainerStarted","Data":"5e9f904504d3b748e89d642586c19f4ff3c5a4cae35d0c5253d16b408dbe68cd"}
Sep 10 21:18:11.755417 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.755395    2573 generic.go:356] "Generic (PLEG): container finished" podID="a8c2eab6-edeb-497b-9674-bae502e2e029" containerID="bdd1315375d9d52e5fa17dc49593b9c84b88a390d3c95c6354eb15ef3a1a716d" exitCode=0
Sep 10 21:18:11.755532 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.755501    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9" event={"ID":"a8c2eab6-edeb-497b-9674-bae502e2e029","Type":"ContainerDied","Data":"bdd1315375d9d52e5fa17dc49593b9c84b88a390d3c95c6354eb15ef3a1a716d"}
Sep 10 21:18:11.755644 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.755540    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9" event={"ID":"a8c2eab6-edeb-497b-9674-bae502e2e029","Type":"ContainerStarted","Data":"2371bd9e1a6b392236e5fdfd1b12342031d0bc949f71b3a8ed50a69685677158"}
Sep 10 21:18:11.757259 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.757235    2573 generic.go:356] "Generic (PLEG): container finished" podID="3c7c216e-033d-4708-81bb-ffeac2ece578" containerID="7545ba5c0f3c5bdd4884c119f90dd725448ae1e1bb5d7175f1ea747fe4e2f7c8" exitCode=0
Sep 10 21:18:11.757473 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.757312    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7" event={"ID":"3c7c216e-033d-4708-81bb-ffeac2ece578","Type":"ContainerDied","Data":"7545ba5c0f3c5bdd4884c119f90dd725448ae1e1bb5d7175f1ea747fe4e2f7c8"}
Sep 10 21:18:11.757473 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:11.757348    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7" event={"ID":"3c7c216e-033d-4708-81bb-ffeac2ece578","Type":"ContainerStarted","Data":"2d57d67d6ebb18db8a07d9d30d2e58455e5c904392ff335ad1f12d96b06410ec"}
Sep 10 21:18:12.766399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:12.766366    2573 generic.go:356] "Generic (PLEG): container finished" podID="c2d88a40-623e-4128-a05b-93cd7499a5bb" containerID="d9a66dc387744b1356782fb3f2ccde854e6df70016db57785bf24e6b1925862a" exitCode=0
Sep 10 21:18:12.766885 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:12.766444    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz" event={"ID":"c2d88a40-623e-4128-a05b-93cd7499a5bb","Type":"ContainerDied","Data":"d9a66dc387744b1356782fb3f2ccde854e6df70016db57785bf24e6b1925862a"}
Sep 10 21:18:13.775418 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:13.775379    2573 generic.go:356] "Generic (PLEG): container finished" podID="c7dd40f6-d94d-4857-a8ad-630c4406af18" containerID="07ee6b0e92d9b3543d2db9f071622226c3bade7bb1e610916d28ffd9bfb09985" exitCode=0
Sep 10 21:18:13.775934 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:13.775464    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44" event={"ID":"c7dd40f6-d94d-4857-a8ad-630c4406af18","Type":"ContainerDied","Data":"07ee6b0e92d9b3543d2db9f071622226c3bade7bb1e610916d28ffd9bfb09985"}
Sep 10 21:18:13.777771 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:13.777749    2573 generic.go:356] "Generic (PLEG): container finished" podID="c2d88a40-623e-4128-a05b-93cd7499a5bb" containerID="5a735641137fab56cf0688552907773e77d399bec2ccd774c80d83181ada832a" exitCode=0
Sep 10 21:18:13.777873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:13.777806    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz" event={"ID":"c2d88a40-623e-4128-a05b-93cd7499a5bb","Type":"ContainerDied","Data":"5a735641137fab56cf0688552907773e77d399bec2ccd774c80d83181ada832a"}
Sep 10 21:18:13.779785 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:13.779750    2573 generic.go:356] "Generic (PLEG): container finished" podID="a8c2eab6-edeb-497b-9674-bae502e2e029" containerID="1e2b440fe21381d7ee060e4f27c334426943a189753c047fda7f669ca4880667" exitCode=0
Sep 10 21:18:13.779900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:13.779793    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9" event={"ID":"a8c2eab6-edeb-497b-9674-bae502e2e029","Type":"ContainerDied","Data":"1e2b440fe21381d7ee060e4f27c334426943a189753c047fda7f669ca4880667"}
Sep 10 21:18:13.781842 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:13.781817    2573 generic.go:356] "Generic (PLEG): container finished" podID="3c7c216e-033d-4708-81bb-ffeac2ece578" containerID="9b261be641d0a10221ff4aafdfea4683b598fb0ab6e4ccd01b39061bcaf52080" exitCode=0
Sep 10 21:18:13.781948 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:13.781897    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7" event={"ID":"3c7c216e-033d-4708-81bb-ffeac2ece578","Type":"ContainerDied","Data":"9b261be641d0a10221ff4aafdfea4683b598fb0ab6e4ccd01b39061bcaf52080"}
Sep 10 21:18:14.789038 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:14.789002    2573 generic.go:356] "Generic (PLEG): container finished" podID="a8c2eab6-edeb-497b-9674-bae502e2e029" containerID="9a7ade4ea72330ec3e43a2429cbb77120bae3f5b264f8573603ca5434589db3d" exitCode=0
Sep 10 21:18:14.789484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:14.789088    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9" event={"ID":"a8c2eab6-edeb-497b-9674-bae502e2e029","Type":"ContainerDied","Data":"9a7ade4ea72330ec3e43a2429cbb77120bae3f5b264f8573603ca5434589db3d"}
Sep 10 21:18:14.791247 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:14.791221    2573 generic.go:356] "Generic (PLEG): container finished" podID="3c7c216e-033d-4708-81bb-ffeac2ece578" containerID="c956b7d8e38a142f477161aec3da1a1e6657be681e0e86aa4f729d3ebc27334e" exitCode=0
Sep 10 21:18:14.791385 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:14.791299    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7" event={"ID":"3c7c216e-033d-4708-81bb-ffeac2ece578","Type":"ContainerDied","Data":"c956b7d8e38a142f477161aec3da1a1e6657be681e0e86aa4f729d3ebc27334e"}
Sep 10 21:18:14.793507 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:14.793486    2573 generic.go:356] "Generic (PLEG): container finished" podID="c7dd40f6-d94d-4857-a8ad-630c4406af18" containerID="beca7eebbba5d332fde3ddb2dfaca35430285567d522b0d3ebd24dde11bd9020" exitCode=0
Sep 10 21:18:14.793616 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:14.793560    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44" event={"ID":"c7dd40f6-d94d-4857-a8ad-630c4406af18","Type":"ContainerDied","Data":"beca7eebbba5d332fde3ddb2dfaca35430285567d522b0d3ebd24dde11bd9020"}
Sep 10 21:18:14.923098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:14.923070    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:15.042723 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.042616    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-bundle\") pod \"c2d88a40-623e-4128-a05b-93cd7499a5bb\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") "
Sep 10 21:18:15.042723 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.042709    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-util\") pod \"c2d88a40-623e-4128-a05b-93cd7499a5bb\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") "
Sep 10 21:18:15.042962 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.042821    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjmm7\" (UniqueName: \"kubernetes.io/projected/c2d88a40-623e-4128-a05b-93cd7499a5bb-kube-api-access-fjmm7\") pod \"c2d88a40-623e-4128-a05b-93cd7499a5bb\" (UID: \"c2d88a40-623e-4128-a05b-93cd7499a5bb\") "
Sep 10 21:18:15.043224 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.043192    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-bundle" (OuterVolumeSpecName: "bundle") pod "c2d88a40-623e-4128-a05b-93cd7499a5bb" (UID: "c2d88a40-623e-4128-a05b-93cd7499a5bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:15.044910 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.044885    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2d88a40-623e-4128-a05b-93cd7499a5bb-kube-api-access-fjmm7" (OuterVolumeSpecName: "kube-api-access-fjmm7") pod "c2d88a40-623e-4128-a05b-93cd7499a5bb" (UID: "c2d88a40-623e-4128-a05b-93cd7499a5bb"). InnerVolumeSpecName "kube-api-access-fjmm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:18:15.048458 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.048413    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-util" (OuterVolumeSpecName: "util") pod "c2d88a40-623e-4128-a05b-93cd7499a5bb" (UID: "c2d88a40-623e-4128-a05b-93cd7499a5bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:15.144042 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.144005    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-bundle\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:15.144042 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.144037    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2d88a40-623e-4128-a05b-93cd7499a5bb-util\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:15.144042 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.144046    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjmm7\" (UniqueName: \"kubernetes.io/projected/c2d88a40-623e-4128-a05b-93cd7499a5bb-kube-api-access-fjmm7\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:15.800551 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.800520    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz"
Sep 10 21:18:15.800551 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.800536    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7bfpdz" event={"ID":"c2d88a40-623e-4128-a05b-93cd7499a5bb","Type":"ContainerDied","Data":"5e9f904504d3b748e89d642586c19f4ff3c5a4cae35d0c5253d16b408dbe68cd"}
Sep 10 21:18:15.801017 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.800582    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e9f904504d3b748e89d642586c19f4ff3c5a4cae35d0c5253d16b408dbe68cd"
Sep 10 21:18:15.953965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.953934    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:15.994560 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.994533    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:15.997907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:15.997882    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:16.052383 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.052297    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lkfp\" (UniqueName: \"kubernetes.io/projected/c7dd40f6-d94d-4857-a8ad-630c4406af18-kube-api-access-5lkfp\") pod \"c7dd40f6-d94d-4857-a8ad-630c4406af18\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") "
Sep 10 21:18:16.052383 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.052353    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-bundle\") pod \"c7dd40f6-d94d-4857-a8ad-630c4406af18\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") "
Sep 10 21:18:16.052589 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.052401    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kg6qq\" (UniqueName: \"kubernetes.io/projected/3c7c216e-033d-4708-81bb-ffeac2ece578-kube-api-access-kg6qq\") pod \"3c7c216e-033d-4708-81bb-ffeac2ece578\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") "
Sep 10 21:18:16.052589 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.052482    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-util\") pod \"3c7c216e-033d-4708-81bb-ffeac2ece578\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") "
Sep 10 21:18:16.052589 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.052510    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94tlw\" (UniqueName: \"kubernetes.io/projected/a8c2eab6-edeb-497b-9674-bae502e2e029-kube-api-access-94tlw\") pod \"a8c2eab6-edeb-497b-9674-bae502e2e029\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") "
Sep 10 21:18:16.052589 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.052544    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-util\") pod \"a8c2eab6-edeb-497b-9674-bae502e2e029\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") "
Sep 10 21:18:16.052589 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.052569    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-bundle\") pod \"a8c2eab6-edeb-497b-9674-bae502e2e029\" (UID: \"a8c2eab6-edeb-497b-9674-bae502e2e029\") "
Sep 10 21:18:16.052873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.052607    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-bundle\") pod \"3c7c216e-033d-4708-81bb-ffeac2ece578\" (UID: \"3c7c216e-033d-4708-81bb-ffeac2ece578\") "
Sep 10 21:18:16.052873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.052632    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-util\") pod \"c7dd40f6-d94d-4857-a8ad-630c4406af18\" (UID: \"c7dd40f6-d94d-4857-a8ad-630c4406af18\") "
Sep 10 21:18:16.053683 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.053652    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-bundle" (OuterVolumeSpecName: "bundle") pod "c7dd40f6-d94d-4857-a8ad-630c4406af18" (UID: "c7dd40f6-d94d-4857-a8ad-630c4406af18"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:16.053903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.053870    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-bundle" (OuterVolumeSpecName: "bundle") pod "3c7c216e-033d-4708-81bb-ffeac2ece578" (UID: "3c7c216e-033d-4708-81bb-ffeac2ece578"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:16.054052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.054021    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-bundle" (OuterVolumeSpecName: "bundle") pod "a8c2eab6-edeb-497b-9674-bae502e2e029" (UID: "a8c2eab6-edeb-497b-9674-bae502e2e029"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:16.054941 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.054909    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c7c216e-033d-4708-81bb-ffeac2ece578-kube-api-access-kg6qq" (OuterVolumeSpecName: "kube-api-access-kg6qq") pod "3c7c216e-033d-4708-81bb-ffeac2ece578" (UID: "3c7c216e-033d-4708-81bb-ffeac2ece578"). InnerVolumeSpecName "kube-api-access-kg6qq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:18:16.055293 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.055262    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c2eab6-edeb-497b-9674-bae502e2e029-kube-api-access-94tlw" (OuterVolumeSpecName: "kube-api-access-94tlw") pod "a8c2eab6-edeb-497b-9674-bae502e2e029" (UID: "a8c2eab6-edeb-497b-9674-bae502e2e029"). InnerVolumeSpecName "kube-api-access-94tlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:18:16.055388 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.055351    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7dd40f6-d94d-4857-a8ad-630c4406af18-kube-api-access-5lkfp" (OuterVolumeSpecName: "kube-api-access-5lkfp") pod "c7dd40f6-d94d-4857-a8ad-630c4406af18" (UID: "c7dd40f6-d94d-4857-a8ad-630c4406af18"). InnerVolumeSpecName "kube-api-access-5lkfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:18:16.059062 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.059035    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-util" (OuterVolumeSpecName: "util") pod "c7dd40f6-d94d-4857-a8ad-630c4406af18" (UID: "c7dd40f6-d94d-4857-a8ad-630c4406af18"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:16.061126 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.061096    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-util" (OuterVolumeSpecName: "util") pod "3c7c216e-033d-4708-81bb-ffeac2ece578" (UID: "3c7c216e-033d-4708-81bb-ffeac2ece578"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:16.061591 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.061557    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-util" (OuterVolumeSpecName: "util") pod "a8c2eab6-edeb-497b-9674-bae502e2e029" (UID: "a8c2eab6-edeb-497b-9674-bae502e2e029"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:16.154222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.154176    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-util\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:16.154222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.154211    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94tlw\" (UniqueName: \"kubernetes.io/projected/a8c2eab6-edeb-497b-9674-bae502e2e029-kube-api-access-94tlw\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:16.154222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.154221    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-util\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:16.154222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.154232    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8c2eab6-edeb-497b-9674-bae502e2e029-bundle\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:16.154486 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.154242    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c7c216e-033d-4708-81bb-ffeac2ece578-bundle\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:16.154486 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.154249    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-util\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:16.154486 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.154256    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lkfp\" (UniqueName: \"kubernetes.io/projected/c7dd40f6-d94d-4857-a8ad-630c4406af18-kube-api-access-5lkfp\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:16.154486 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.154264    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7dd40f6-d94d-4857-a8ad-630c4406af18-bundle\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:16.154486 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.154274    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg6qq\" (UniqueName: \"kubernetes.io/projected/3c7c216e-033d-4708-81bb-ffeac2ece578-kube-api-access-kg6qq\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:18:16.807928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.807893    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44"
Sep 10 21:18:16.807928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.807910    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctqj44" event={"ID":"c7dd40f6-d94d-4857-a8ad-630c4406af18","Type":"ContainerDied","Data":"a76544b6b89bfaba4206b7c4e90aca01217bee15dad3d10e5e1d91d71da75f10"}
Sep 10 21:18:16.808434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.807967    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a76544b6b89bfaba4206b7c4e90aca01217bee15dad3d10e5e1d91d71da75f10"
Sep 10 21:18:16.810051 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.810023    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9" event={"ID":"a8c2eab6-edeb-497b-9674-bae502e2e029","Type":"ContainerDied","Data":"2371bd9e1a6b392236e5fdfd1b12342031d0bc949f71b3a8ed50a69685677158"}
Sep 10 21:18:16.810235 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.810149    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2371bd9e1a6b392236e5fdfd1b12342031d0bc949f71b3a8ed50a69685677158"
Sep 10 21:18:16.810235 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.810032    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at8tb9"
Sep 10 21:18:16.812263 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.812232    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7" event={"ID":"3c7c216e-033d-4708-81bb-ffeac2ece578","Type":"ContainerDied","Data":"2d57d67d6ebb18db8a07d9d30d2e58455e5c904392ff335ad1f12d96b06410ec"}
Sep 10 21:18:16.812368 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.812283    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d57d67d6ebb18db8a07d9d30d2e58455e5c904392ff335ad1f12d96b06410ec"
Sep 10 21:18:16.812368 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:16.812304    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hhqr7"
Sep 10 21:18:32.642494 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642454    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"]
Sep 10 21:18:32.642918 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642899    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8c2eab6-edeb-497b-9674-bae502e2e029" containerName="pull"
Sep 10 21:18:32.642965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642920    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8c2eab6-edeb-497b-9674-bae502e2e029" containerName="pull"
Sep 10 21:18:32.642965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642932    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8c2eab6-edeb-497b-9674-bae502e2e029" containerName="extract"
Sep 10 21:18:32.642965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642937    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8c2eab6-edeb-497b-9674-bae502e2e029" containerName="extract"
Sep 10 21:18:32.642965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642944    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7dd40f6-d94d-4857-a8ad-630c4406af18" containerName="extract"
Sep 10 21:18:32.642965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642952    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7dd40f6-d94d-4857-a8ad-630c4406af18" containerName="extract"
Sep 10 21:18:32.642965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642961    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7c216e-033d-4708-81bb-ffeac2ece578" containerName="util"
Sep 10 21:18:32.642965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642966    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7c216e-033d-4708-81bb-ffeac2ece578" containerName="util"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642973    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7dd40f6-d94d-4857-a8ad-630c4406af18" containerName="util"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642978    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7dd40f6-d94d-4857-a8ad-630c4406af18" containerName="util"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642990    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d88a40-623e-4128-a05b-93cd7499a5bb" containerName="util"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.642995    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d88a40-623e-4128-a05b-93cd7499a5bb" containerName="util"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643001    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d88a40-623e-4128-a05b-93cd7499a5bb" containerName="pull"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643006    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d88a40-623e-4128-a05b-93cd7499a5bb" containerName="pull"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643013    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8c2eab6-edeb-497b-9674-bae502e2e029" containerName="util"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643018    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8c2eab6-edeb-497b-9674-bae502e2e029" containerName="util"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643023    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7dd40f6-d94d-4857-a8ad-630c4406af18" containerName="pull"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643028    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7dd40f6-d94d-4857-a8ad-630c4406af18" containerName="pull"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643033    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7c216e-033d-4708-81bb-ffeac2ece578" containerName="extract"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643038    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7c216e-033d-4708-81bb-ffeac2ece578" containerName="extract"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643044    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d88a40-623e-4128-a05b-93cd7499a5bb" containerName="extract"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643051    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d88a40-623e-4128-a05b-93cd7499a5bb" containerName="extract"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643058    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7c216e-033d-4708-81bb-ffeac2ece578" containerName="pull"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643062    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7c216e-033d-4708-81bb-ffeac2ece578" containerName="pull"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643111    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2d88a40-623e-4128-a05b-93cd7499a5bb" containerName="extract"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643119    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7dd40f6-d94d-4857-a8ad-630c4406af18" containerName="extract"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643127    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c7c216e-033d-4708-81bb-ffeac2ece578" containerName="extract"
Sep 10 21:18:32.643164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.643132    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8c2eab6-edeb-497b-9674-bae502e2e029" containerName="extract"
Sep 10 21:18:32.645941 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.645925    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:32.647962 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.647937    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 21:18:32.648100 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.648036    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 21:18:32.648667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.648640    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 21:18:32.648667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.648654    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5x6vr\""
Sep 10 21:18:32.648667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.648663    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 21:18:32.655148 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.655123    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"]
Sep 10 21:18:32.808736 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.808681    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b887163-5548-4810-b818-aaf178145ed4-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-tdbzm\" (UID: \"8b887163-5548-4810-b818-aaf178145ed4\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:32.809021 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.808847    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgc46\" (UniqueName: \"kubernetes.io/projected/8b887163-5548-4810-b818-aaf178145ed4-kube-api-access-hgc46\") pod \"kuadrant-console-plugin-fbdd4f445-tdbzm\" (UID: \"8b887163-5548-4810-b818-aaf178145ed4\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:32.809021 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.808892    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8b887163-5548-4810-b818-aaf178145ed4-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-tdbzm\" (UID: \"8b887163-5548-4810-b818-aaf178145ed4\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:32.909520 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.909423    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgc46\" (UniqueName: \"kubernetes.io/projected/8b887163-5548-4810-b818-aaf178145ed4-kube-api-access-hgc46\") pod \"kuadrant-console-plugin-fbdd4f445-tdbzm\" (UID: \"8b887163-5548-4810-b818-aaf178145ed4\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:32.909520 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.909468    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8b887163-5548-4810-b818-aaf178145ed4-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-tdbzm\" (UID: \"8b887163-5548-4810-b818-aaf178145ed4\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:32.909704 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.909561    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b887163-5548-4810-b818-aaf178145ed4-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-tdbzm\" (UID: \"8b887163-5548-4810-b818-aaf178145ed4\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:32.910174 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.910148    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8b887163-5548-4810-b818-aaf178145ed4-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-tdbzm\" (UID: \"8b887163-5548-4810-b818-aaf178145ed4\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:32.912026 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.911999    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b887163-5548-4810-b818-aaf178145ed4-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-tdbzm\" (UID: \"8b887163-5548-4810-b818-aaf178145ed4\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:32.923983 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.923946    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgc46\" (UniqueName: \"kubernetes.io/projected/8b887163-5548-4810-b818-aaf178145ed4-kube-api-access-hgc46\") pod \"kuadrant-console-plugin-fbdd4f445-tdbzm\" (UID: \"8b887163-5548-4810-b818-aaf178145ed4\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:32.955832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:32.955764    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"
Sep 10 21:18:33.230890 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:33.230859    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm"]
Sep 10 21:18:33.233076 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:18:33.233048    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b887163_5548_4810_b818_aaf178145ed4.slice/crio-ada581b3412f73eb7287b46e08d88a3ebb3023918d17b980ad3df2b0cfd2ee39 WatchSource:0}: Error finding container ada581b3412f73eb7287b46e08d88a3ebb3023918d17b980ad3df2b0cfd2ee39: Status 404 returned error can't find the container with id ada581b3412f73eb7287b46e08d88a3ebb3023918d17b980ad3df2b0cfd2ee39
Sep 10 21:18:33.897472 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:33.897436    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm" event={"ID":"8b887163-5548-4810-b818-aaf178145ed4","Type":"ContainerStarted","Data":"ada581b3412f73eb7287b46e08d88a3ebb3023918d17b980ad3df2b0cfd2ee39"}
Sep 10 21:18:37.921576 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:37.921537    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm" event={"ID":"8b887163-5548-4810-b818-aaf178145ed4","Type":"ContainerStarted","Data":"3b845995cd0e2c7799fa78121cbce91dbb8d6f7a6e2a34016113c3a1e226cc6d"}
Sep 10 21:18:37.937555 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:18:37.937499    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tdbzm" podStartSLOduration=1.370756297 podStartE2EDuration="5.93748345s" totalImagesPullingTime="4.566727153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:37.935943593 +0000 UTC m=+493.869610408" watchObservedRunningTime="2026-09-10 21:18:37.93748345 +0000 UTC m=+493.871150273"
Sep 10 21:19:26.579946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.579908    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:19:26.582200 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.582184    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74"
Sep 10 21:19:26.584038 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.584017    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 21:19:26.592267 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.592242    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:19:26.628239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.628201    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:19:26.645414 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.645384    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wndqt\" (UniqueName: \"kubernetes.io/projected/220ecdaf-58d6-4f99-bedc-eb3ed2906d42-kube-api-access-wndqt\") pod \"limitador-limitador-84994f7ffc-4nr74\" (UID: \"220ecdaf-58d6-4f99-bedc-eb3ed2906d42\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74"
Sep 10 21:19:26.645572 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.645427    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/220ecdaf-58d6-4f99-bedc-eb3ed2906d42-config-file\") pod \"limitador-limitador-84994f7ffc-4nr74\" (UID: \"220ecdaf-58d6-4f99-bedc-eb3ed2906d42\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74"
Sep 10 21:19:26.746921 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.746878    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wndqt\" (UniqueName: \"kubernetes.io/projected/220ecdaf-58d6-4f99-bedc-eb3ed2906d42-kube-api-access-wndqt\") pod \"limitador-limitador-84994f7ffc-4nr74\" (UID: \"220ecdaf-58d6-4f99-bedc-eb3ed2906d42\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74"
Sep 10 21:19:26.747138 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.746934    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/220ecdaf-58d6-4f99-bedc-eb3ed2906d42-config-file\") pod \"limitador-limitador-84994f7ffc-4nr74\" (UID: \"220ecdaf-58d6-4f99-bedc-eb3ed2906d42\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74"
Sep 10 21:19:26.747680 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.747657    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/220ecdaf-58d6-4f99-bedc-eb3ed2906d42-config-file\") pod \"limitador-limitador-84994f7ffc-4nr74\" (UID: \"220ecdaf-58d6-4f99-bedc-eb3ed2906d42\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74"
Sep 10 21:19:26.756403 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.756376    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wndqt\" (UniqueName: \"kubernetes.io/projected/220ecdaf-58d6-4f99-bedc-eb3ed2906d42-kube-api-access-wndqt\") pod \"limitador-limitador-84994f7ffc-4nr74\" (UID: \"220ecdaf-58d6-4f99-bedc-eb3ed2906d42\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74"
Sep 10 21:19:26.892683 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.892581    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74"
Sep 10 21:19:26.995000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.994967    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x4klq"]
Sep 10 21:19:26.997549 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.997531    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq"
Sep 10 21:19:26.999764 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:26.999739    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d5rjx\""
Sep 10 21:19:27.005938 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:27.005915    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x4klq"]
Sep 10 21:19:27.150911 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:27.150832    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfq99\" (UniqueName: \"kubernetes.io/projected/c84715d9-8a7c-4c2b-8e6e-5a1031fac957-kube-api-access-wfq99\") pod \"authorino-7bcfd8f9cd-x4klq\" (UID: \"c84715d9-8a7c-4c2b-8e6e-5a1031fac957\") " pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq"
Sep 10 21:19:27.162429 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:27.162404    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:19:27.164618 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:19:27.164588    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220ecdaf_58d6_4f99_bedc_eb3ed2906d42.slice/crio-f17dc518d38fb8d471d4d8cb69b4f30fcdf883d1212bcdc9df34047294e64cda WatchSource:0}: Error finding container f17dc518d38fb8d471d4d8cb69b4f30fcdf883d1212bcdc9df34047294e64cda: Status 404 returned error can't find the container with id f17dc518d38fb8d471d4d8cb69b4f30fcdf883d1212bcdc9df34047294e64cda
Sep 10 21:19:27.251620 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:27.251576    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfq99\" (UniqueName: \"kubernetes.io/projected/c84715d9-8a7c-4c2b-8e6e-5a1031fac957-kube-api-access-wfq99\") pod \"authorino-7bcfd8f9cd-x4klq\" (UID: \"c84715d9-8a7c-4c2b-8e6e-5a1031fac957\") " pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq"
Sep 10 21:19:27.259765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:27.259725    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfq99\" (UniqueName: \"kubernetes.io/projected/c84715d9-8a7c-4c2b-8e6e-5a1031fac957-kube-api-access-wfq99\") pod \"authorino-7bcfd8f9cd-x4klq\" (UID: \"c84715d9-8a7c-4c2b-8e6e-5a1031fac957\") " pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq"
Sep 10 21:19:27.307805 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:27.307749    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq"
Sep 10 21:19:27.575226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:27.575200    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x4klq"]
Sep 10 21:19:27.576972 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:19:27.576947    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc84715d9_8a7c_4c2b_8e6e_5a1031fac957.slice/crio-d5e3dcfb03ef4dd8232eb42fe59e3281dec82a2258518443692c1d970e316553 WatchSource:0}: Error finding container d5e3dcfb03ef4dd8232eb42fe59e3281dec82a2258518443692c1d970e316553: Status 404 returned error can't find the container with id d5e3dcfb03ef4dd8232eb42fe59e3281dec82a2258518443692c1d970e316553
Sep 10 21:19:28.172943 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:28.172904    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74" event={"ID":"220ecdaf-58d6-4f99-bedc-eb3ed2906d42","Type":"ContainerStarted","Data":"f17dc518d38fb8d471d4d8cb69b4f30fcdf883d1212bcdc9df34047294e64cda"}
Sep 10 21:19:28.174055 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:28.174032    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq" event={"ID":"c84715d9-8a7c-4c2b-8e6e-5a1031fac957","Type":"ContainerStarted","Data":"d5e3dcfb03ef4dd8232eb42fe59e3281dec82a2258518443692c1d970e316553"}
Sep 10 21:19:30.351148 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:30.351098    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x4klq"]
Sep 10 21:19:31.191615 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:31.191578    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq" event={"ID":"c84715d9-8a7c-4c2b-8e6e-5a1031fac957","Type":"ContainerStarted","Data":"48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9"}
Sep 10 21:19:31.206917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:31.206862    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq" podStartSLOduration=2.60450956 podStartE2EDuration="5.206844803s" totalImagesPullingTime="2.602335243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:31.205258312 +0000 UTC m=+547.138925124" watchObservedRunningTime="2026-09-10 21:19:31.206844803 +0000 UTC m=+547.140511617"
Sep 10 21:19:32.197751 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:32.197676    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq" podUID="c84715d9-8a7c-4c2b-8e6e-5a1031fac957" containerName="authorino" containerID="cri-o://48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9" gracePeriod=30
Sep 10 21:19:32.450078 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:32.450019    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq"
Sep 10 21:19:32.600987 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:32.600942    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfq99\" (UniqueName: \"kubernetes.io/projected/c84715d9-8a7c-4c2b-8e6e-5a1031fac957-kube-api-access-wfq99\") pod \"c84715d9-8a7c-4c2b-8e6e-5a1031fac957\" (UID: \"c84715d9-8a7c-4c2b-8e6e-5a1031fac957\") "
Sep 10 21:19:32.603143 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:32.603110    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c84715d9-8a7c-4c2b-8e6e-5a1031fac957-kube-api-access-wfq99" (OuterVolumeSpecName: "kube-api-access-wfq99") pod "c84715d9-8a7c-4c2b-8e6e-5a1031fac957" (UID: "c84715d9-8a7c-4c2b-8e6e-5a1031fac957"). InnerVolumeSpecName "kube-api-access-wfq99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:19:32.701625 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:32.701546    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfq99\" (UniqueName: \"kubernetes.io/projected/c84715d9-8a7c-4c2b-8e6e-5a1031fac957-kube-api-access-wfq99\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:19:33.204192 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.204152    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74" event={"ID":"220ecdaf-58d6-4f99-bedc-eb3ed2906d42","Type":"ContainerStarted","Data":"6baa3c45ca9204e0461730b3ec6f867319b2ffd601c40eea25813c3bca6c307c"}
Sep 10 21:19:33.204682 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.204225    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74"
Sep 10 21:19:33.205472 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.205449    2573 generic.go:356] "Generic (PLEG): container finished" podID="c84715d9-8a7c-4c2b-8e6e-5a1031fac957" containerID="48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9" exitCode=0
Sep 10 21:19:33.205611 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.205481    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq"
Sep 10 21:19:33.205611 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.205509    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq" event={"ID":"c84715d9-8a7c-4c2b-8e6e-5a1031fac957","Type":"ContainerDied","Data":"48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9"}
Sep 10 21:19:33.205611 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.205545    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-x4klq" event={"ID":"c84715d9-8a7c-4c2b-8e6e-5a1031fac957","Type":"ContainerDied","Data":"d5e3dcfb03ef4dd8232eb42fe59e3281dec82a2258518443692c1d970e316553"}
Sep 10 21:19:33.205611 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.205568    2573 scope.go:117] "RemoveContainer" containerID="48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9"
Sep 10 21:19:33.214562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.214538    2573 scope.go:117] "RemoveContainer" containerID="48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9"
Sep 10 21:19:33.215016 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:19:33.214995    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9\": container with ID starting with 48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9 not found: ID does not exist" containerID="48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9"
Sep 10 21:19:33.215086 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.215027    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9"} err="failed to get container status \"48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9\": rpc error: code = NotFound desc = could not find container \"48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9\": container with ID starting with 48a85a836b64482162f4043d513d2245363841e6963a0cacfde9d16b51360fa9 not found: ID does not exist"
Sep 10 21:19:33.220878 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.220825    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74" podStartSLOduration=2.067057521 podStartE2EDuration="7.220808067s" totalImagesPullingTime="5.153750546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:33.218932524 +0000 UTC m=+549.152599343" watchObservedRunningTime="2026-09-10 21:19:33.220808067 +0000 UTC m=+549.154474876"
Sep 10 21:19:33.233305 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.233265    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x4klq"]
Sep 10 21:19:33.239241 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:33.239210    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x4klq"]
Sep 10 21:19:34.680200 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:34.680167    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c84715d9-8a7c-4c2b-8e6e-5a1031fac957" path="/var/lib/kubelet/pods/c84715d9-8a7c-4c2b-8e6e-5a1031fac957/volumes"
Sep 10 21:19:44.216767 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:44.216734    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-4nr74"
Sep 10 21:19:58.969886 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:58.969844    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-nqk8c"]
Sep 10 21:19:58.970400 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:58.970339    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84715d9-8a7c-4c2b-8e6e-5a1031fac957" containerName="authorino"
Sep 10 21:19:58.970400 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:58.970357    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84715d9-8a7c-4c2b-8e6e-5a1031fac957" containerName="authorino"
Sep 10 21:19:58.970519 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:58.970447    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c84715d9-8a7c-4c2b-8e6e-5a1031fac957" containerName="authorino"
Sep 10 21:19:58.973956 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:58.973939    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nqk8c"
Sep 10 21:19:58.977190 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:58.977169    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d5rjx\""
Sep 10 21:19:58.982238 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:58.982211    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-nqk8c"]
Sep 10 21:19:59.036049 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:59.036010    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsrfb\" (UniqueName: \"kubernetes.io/projected/4a3983f4-d261-4de1-afa7-5cd39faf6745-kube-api-access-tsrfb\") pod \"authorino-557f876f69-nqk8c\" (UID: \"4a3983f4-d261-4de1-afa7-5cd39faf6745\") " pod="kuadrant-system/authorino-557f876f69-nqk8c"
Sep 10 21:19:59.136620 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:59.136580    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsrfb\" (UniqueName: \"kubernetes.io/projected/4a3983f4-d261-4de1-afa7-5cd39faf6745-kube-api-access-tsrfb\") pod \"authorino-557f876f69-nqk8c\" (UID: \"4a3983f4-d261-4de1-afa7-5cd39faf6745\") " pod="kuadrant-system/authorino-557f876f69-nqk8c"
Sep 10 21:19:59.144933 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:59.144906    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsrfb\" (UniqueName: \"kubernetes.io/projected/4a3983f4-d261-4de1-afa7-5cd39faf6745-kube-api-access-tsrfb\") pod \"authorino-557f876f69-nqk8c\" (UID: \"4a3983f4-d261-4de1-afa7-5cd39faf6745\") " pod="kuadrant-system/authorino-557f876f69-nqk8c"
Sep 10 21:19:59.198107 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:59.198072    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-nqk8c"]
Sep 10 21:19:59.198341 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:59.198329    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nqk8c"
Sep 10 21:19:59.472285 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:19:59.472259    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-nqk8c"]
Sep 10 21:19:59.475556 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:19:59.475524    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a3983f4_d261_4de1_afa7_5cd39faf6745.slice/crio-cb904434916611fca41d67385836716e925564c0d0bfde5f9b543c2147371e48 WatchSource:0}: Error finding container cb904434916611fca41d67385836716e925564c0d0bfde5f9b543c2147371e48: Status 404 returned error can't find the container with id cb904434916611fca41d67385836716e925564c0d0bfde5f9b543c2147371e48
Sep 10 21:20:00.346709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:00.346669    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-nqk8c" event={"ID":"4a3983f4-d261-4de1-afa7-5cd39faf6745","Type":"ContainerStarted","Data":"c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9"}
Sep 10 21:20:00.347169 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:00.346719    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-nqk8c" event={"ID":"4a3983f4-d261-4de1-afa7-5cd39faf6745","Type":"ContainerStarted","Data":"cb904434916611fca41d67385836716e925564c0d0bfde5f9b543c2147371e48"}
Sep 10 21:20:00.347169 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:00.346747    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-nqk8c" podUID="4a3983f4-d261-4de1-afa7-5cd39faf6745" containerName="authorino" containerID="cri-o://c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9" gracePeriod=30
Sep 10 21:20:00.362593 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:00.362545    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-nqk8c" podStartSLOduration=1.8848216930000001 podStartE2EDuration="2.362526886s" totalImagesPullingTime="477.705193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:00.360328922 +0000 UTC m=+576.293995736" watchObservedRunningTime="2026-09-10 21:20:00.362526886 +0000 UTC m=+576.296193702"
Sep 10 21:20:00.588716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:00.588692    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nqk8c"
Sep 10 21:20:00.648031 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:00.647940    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsrfb\" (UniqueName: \"kubernetes.io/projected/4a3983f4-d261-4de1-afa7-5cd39faf6745-kube-api-access-tsrfb\") pod \"4a3983f4-d261-4de1-afa7-5cd39faf6745\" (UID: \"4a3983f4-d261-4de1-afa7-5cd39faf6745\") "
Sep 10 21:20:00.650128 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:00.650093    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a3983f4-d261-4de1-afa7-5cd39faf6745-kube-api-access-tsrfb" (OuterVolumeSpecName: "kube-api-access-tsrfb") pod "4a3983f4-d261-4de1-afa7-5cd39faf6745" (UID: "4a3983f4-d261-4de1-afa7-5cd39faf6745"). InnerVolumeSpecName "kube-api-access-tsrfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:20:00.748968 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:00.748933    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsrfb\" (UniqueName: \"kubernetes.io/projected/4a3983f4-d261-4de1-afa7-5cd39faf6745-kube-api-access-tsrfb\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:20:01.352550 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.352509    2573 generic.go:356] "Generic (PLEG): container finished" podID="4a3983f4-d261-4de1-afa7-5cd39faf6745" containerID="c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9" exitCode=0
Sep 10 21:20:01.352991 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.352564    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-nqk8c" event={"ID":"4a3983f4-d261-4de1-afa7-5cd39faf6745","Type":"ContainerDied","Data":"c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9"}
Sep 10 21:20:01.352991 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.352574    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nqk8c"
Sep 10 21:20:01.352991 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.352604    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-nqk8c" event={"ID":"4a3983f4-d261-4de1-afa7-5cd39faf6745","Type":"ContainerDied","Data":"cb904434916611fca41d67385836716e925564c0d0bfde5f9b543c2147371e48"}
Sep 10 21:20:01.352991 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.352626    2573 scope.go:117] "RemoveContainer" containerID="c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9"
Sep 10 21:20:01.361482 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.361458    2573 scope.go:117] "RemoveContainer" containerID="c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9"
Sep 10 21:20:01.361786 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:20:01.361748    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9\": container with ID starting with c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9 not found: ID does not exist" containerID="c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9"
Sep 10 21:20:01.361854 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.361801    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9"} err="failed to get container status \"c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9\": rpc error: code = NotFound desc = could not find container \"c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9\": container with ID starting with c03092dc6dc7244972387447b588fb655b86117ab2173e50ebef7f427296e5c9 not found: ID does not exist"
Sep 10 21:20:01.369351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.369320    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-nqk8c"]
Sep 10 21:20:01.375684 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.375661    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-nqk8c"]
Sep 10 21:20:01.571389 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.571354    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c8759f49-lc2rp"]
Sep 10 21:20:01.571747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.571725    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a3983f4-d261-4de1-afa7-5cd39faf6745" containerName="authorino"
Sep 10 21:20:01.571818 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.571749    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a3983f4-d261-4de1-afa7-5cd39faf6745" containerName="authorino"
Sep 10 21:20:01.571858 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.571821    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a3983f4-d261-4de1-afa7-5cd39faf6745" containerName="authorino"
Sep 10 21:20:01.574788 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.574757    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c8759f49-lc2rp"
Sep 10 21:20:01.576949 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.576926    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 21:20:01.576949 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.576927    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d5rjx\""
Sep 10 21:20:01.584872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.584846    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c8759f49-lc2rp"]
Sep 10 21:20:01.657251 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.657147    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57v6s\" (UniqueName: \"kubernetes.io/projected/367196de-a050-496d-a13a-bebae099f1a5-kube-api-access-57v6s\") pod \"authorino-5c8759f49-lc2rp\" (UID: \"367196de-a050-496d-a13a-bebae099f1a5\") " pod="kuadrant-system/authorino-5c8759f49-lc2rp"
Sep 10 21:20:01.657251 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.657212    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/367196de-a050-496d-a13a-bebae099f1a5-tls-cert\") pod \"authorino-5c8759f49-lc2rp\" (UID: \"367196de-a050-496d-a13a-bebae099f1a5\") " pod="kuadrant-system/authorino-5c8759f49-lc2rp"
Sep 10 21:20:01.757742 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.757707    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57v6s\" (UniqueName: \"kubernetes.io/projected/367196de-a050-496d-a13a-bebae099f1a5-kube-api-access-57v6s\") pod \"authorino-5c8759f49-lc2rp\" (UID: \"367196de-a050-496d-a13a-bebae099f1a5\") " pod="kuadrant-system/authorino-5c8759f49-lc2rp"
Sep 10 21:20:01.757943 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.757760    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/367196de-a050-496d-a13a-bebae099f1a5-tls-cert\") pod \"authorino-5c8759f49-lc2rp\" (UID: \"367196de-a050-496d-a13a-bebae099f1a5\") " pod="kuadrant-system/authorino-5c8759f49-lc2rp"
Sep 10 21:20:01.760314 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.760285    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/367196de-a050-496d-a13a-bebae099f1a5-tls-cert\") pod \"authorino-5c8759f49-lc2rp\" (UID: \"367196de-a050-496d-a13a-bebae099f1a5\") " pod="kuadrant-system/authorino-5c8759f49-lc2rp"
Sep 10 21:20:01.765196 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.765166    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57v6s\" (UniqueName: \"kubernetes.io/projected/367196de-a050-496d-a13a-bebae099f1a5-kube-api-access-57v6s\") pod \"authorino-5c8759f49-lc2rp\" (UID: \"367196de-a050-496d-a13a-bebae099f1a5\") " pod="kuadrant-system/authorino-5c8759f49-lc2rp"
Sep 10 21:20:01.884981 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:01.884941    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c8759f49-lc2rp"
Sep 10 21:20:02.155225 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:02.155134    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c8759f49-lc2rp"]
Sep 10 21:20:02.157739 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:20:02.157702    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod367196de_a050_496d_a13a_bebae099f1a5.slice/crio-c9c3734ebf09d8a53742fcf35ef877103ad22ca2a2cc68f8c64a7957e37f3b59 WatchSource:0}: Error finding container c9c3734ebf09d8a53742fcf35ef877103ad22ca2a2cc68f8c64a7957e37f3b59: Status 404 returned error can't find the container with id c9c3734ebf09d8a53742fcf35ef877103ad22ca2a2cc68f8c64a7957e37f3b59
Sep 10 21:20:02.362963 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:02.362930    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c8759f49-lc2rp" event={"ID":"367196de-a050-496d-a13a-bebae099f1a5","Type":"ContainerStarted","Data":"c9c3734ebf09d8a53742fcf35ef877103ad22ca2a2cc68f8c64a7957e37f3b59"}
Sep 10 21:20:02.679849 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:02.679818    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a3983f4-d261-4de1-afa7-5cd39faf6745" path="/var/lib/kubelet/pods/4a3983f4-d261-4de1-afa7-5cd39faf6745/volumes"
Sep 10 21:20:03.370792 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:03.370742    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c8759f49-lc2rp" event={"ID":"367196de-a050-496d-a13a-bebae099f1a5","Type":"ContainerStarted","Data":"55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1"}
Sep 10 21:20:03.388808 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:03.388720    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5c8759f49-lc2rp" podStartSLOduration=1.951329216 podStartE2EDuration="2.388703043s" totalImagesPullingTime="437.373827ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:03.386249626 +0000 UTC m=+579.319916442" watchObservedRunningTime="2026-09-10 21:20:03.388703043 +0000 UTC m=+579.322369858"
Sep 10 21:20:24.469066 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:24.469033    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:20:24.473311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:24.473286    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:20:43.472925 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:20:43.472886    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:21:04.262519 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:04.262482    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:21:23.823816 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:23.823761    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:21:30.718906 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:30.718866    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:21:34.612537 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.612504    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"]
Sep 10 21:21:34.615874 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.615855    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.618184 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.618156    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 21:21:34.618992 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.618969    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 21:21:34.619082 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.618974    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sprxz\""
Sep 10 21:21:34.619082 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.619015    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 21:21:34.627644 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.627617    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"]
Sep 10 21:21:34.673549 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.673515    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.673549 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.673550    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrr4j\" (UniqueName: \"kubernetes.io/projected/caee352d-a74c-4581-a25c-eb44823eb5c6-kube-api-access-rrr4j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.673832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.673601    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.673832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.673655    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.673832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.673735    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.673832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.673796    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/caee352d-a74c-4581-a25c-eb44823eb5c6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.774992 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.774957    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.775189 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.774996    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/caee352d-a74c-4581-a25c-eb44823eb5c6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.775189 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.775029    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.775189 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.775149    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrr4j\" (UniqueName: \"kubernetes.io/projected/caee352d-a74c-4581-a25c-eb44823eb5c6-kube-api-access-rrr4j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.775355 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.775217    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.775355 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.775308    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.775464 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.775410    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.775520 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.775463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.775675 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.775652    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.777404 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.777386    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/caee352d-a74c-4581-a25c-eb44823eb5c6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.777861 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.777839    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/caee352d-a74c-4581-a25c-eb44823eb5c6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.783761 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.783731    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrr4j\" (UniqueName: \"kubernetes.io/projected/caee352d-a74c-4581-a25c-eb44823eb5c6-kube-api-access-rrr4j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hgpnd\" (UID: \"caee352d-a74c-4581-a25c-eb44823eb5c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:34.926535 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:34.926427    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:35.198204 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:35.198170    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"]
Sep 10 21:21:35.199411 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:21:35.199386    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaee352d_a74c_4581_a25c_eb44823eb5c6.slice/crio-54df8243853f83168bf0328bdc8ceb160e2827d1a357e30af6fdc955ce2649aa WatchSource:0}: Error finding container 54df8243853f83168bf0328bdc8ceb160e2827d1a357e30af6fdc955ce2649aa: Status 404 returned error can't find the container with id 54df8243853f83168bf0328bdc8ceb160e2827d1a357e30af6fdc955ce2649aa
Sep 10 21:21:35.200785 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:35.200754    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:21:35.724912 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:35.724873    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:21:35.852838 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:35.852788    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd" event={"ID":"caee352d-a74c-4581-a25c-eb44823eb5c6","Type":"ContainerStarted","Data":"54df8243853f83168bf0328bdc8ceb160e2827d1a357e30af6fdc955ce2649aa"}
Sep 10 21:21:36.859972 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:36.859932    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd" event={"ID":"caee352d-a74c-4581-a25c-eb44823eb5c6","Type":"ContainerStarted","Data":"663149a7380e02e591924f38e6f9ac9c8d2b1292f130de597d6161371a24d14c"}
Sep 10 21:21:36.880311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:36.880256    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd" podStartSLOduration=1.81924066 podStartE2EDuration="2.880238997s" totalImagesPullingTime="1.060998337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:21:36.878391432 +0000 UTC m=+672.812058249" watchObservedRunningTime="2026-09-10 21:21:36.880238997 +0000 UTC m=+672.813905812"
Sep 10 21:21:43.614541 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:43.614497    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:21:44.927405 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:44.927360    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:44.927895 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:44.927423    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:44.940231 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:44.940203    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:45.918598 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:45.918565    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hgpnd"
Sep 10 21:21:56.011309 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:21:56.011275    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:22:05.324342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:05.324308    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:22:51.868826 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:51.868759    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6454784d76-w2h5d"]
Sep 10 21:22:51.871215 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:51.871199    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6454784d76-w2h5d"
Sep 10 21:22:51.882328 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:51.882306    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6454784d76-w2h5d"]
Sep 10 21:22:52.064021 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:52.063974    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tlnp\" (UniqueName: \"kubernetes.io/projected/b82d9d86-3adf-4478-beee-5f2145c09030-kube-api-access-8tlnp\") pod \"authorino-6454784d76-w2h5d\" (UID: \"b82d9d86-3adf-4478-beee-5f2145c09030\") " pod="kuadrant-system/authorino-6454784d76-w2h5d"
Sep 10 21:22:52.064021 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:52.064027    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b82d9d86-3adf-4478-beee-5f2145c09030-tls-cert\") pod \"authorino-6454784d76-w2h5d\" (UID: \"b82d9d86-3adf-4478-beee-5f2145c09030\") " pod="kuadrant-system/authorino-6454784d76-w2h5d"
Sep 10 21:22:52.165508 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:52.165421    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tlnp\" (UniqueName: \"kubernetes.io/projected/b82d9d86-3adf-4478-beee-5f2145c09030-kube-api-access-8tlnp\") pod \"authorino-6454784d76-w2h5d\" (UID: \"b82d9d86-3adf-4478-beee-5f2145c09030\") " pod="kuadrant-system/authorino-6454784d76-w2h5d"
Sep 10 21:22:52.165508 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:52.165466    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b82d9d86-3adf-4478-beee-5f2145c09030-tls-cert\") pod \"authorino-6454784d76-w2h5d\" (UID: \"b82d9d86-3adf-4478-beee-5f2145c09030\") " pod="kuadrant-system/authorino-6454784d76-w2h5d"
Sep 10 21:22:52.167924 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:52.167902    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b82d9d86-3adf-4478-beee-5f2145c09030-tls-cert\") pod \"authorino-6454784d76-w2h5d\" (UID: \"b82d9d86-3adf-4478-beee-5f2145c09030\") " pod="kuadrant-system/authorino-6454784d76-w2h5d"
Sep 10 21:22:52.175192 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:52.175168    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tlnp\" (UniqueName: \"kubernetes.io/projected/b82d9d86-3adf-4478-beee-5f2145c09030-kube-api-access-8tlnp\") pod \"authorino-6454784d76-w2h5d\" (UID: \"b82d9d86-3adf-4478-beee-5f2145c09030\") " pod="kuadrant-system/authorino-6454784d76-w2h5d"
Sep 10 21:22:52.179934 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:52.179907    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6454784d76-w2h5d"
Sep 10 21:22:52.451442 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:52.451417    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6454784d76-w2h5d"]
Sep 10 21:22:52.453184 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:22:52.453159    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb82d9d86_3adf_4478_beee_5f2145c09030.slice/crio-372777fd125e52a2f529f12b07f481ff421705c8eac893b54a4f6e7c9899a57e WatchSource:0}: Error finding container 372777fd125e52a2f529f12b07f481ff421705c8eac893b54a4f6e7c9899a57e: Status 404 returned error can't find the container with id 372777fd125e52a2f529f12b07f481ff421705c8eac893b54a4f6e7c9899a57e
Sep 10 21:22:53.256626 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:53.256592    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6454784d76-w2h5d" event={"ID":"b82d9d86-3adf-4478-beee-5f2145c09030","Type":"ContainerStarted","Data":"372777fd125e52a2f529f12b07f481ff421705c8eac893b54a4f6e7c9899a57e"}
Sep 10 21:22:54.265222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.265176    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6454784d76-w2h5d" event={"ID":"b82d9d86-3adf-4478-beee-5f2145c09030","Type":"ContainerStarted","Data":"bbbc7ab5939f53ccab74d7ec3a74ff0fd30a9d479f403ad8506d04d4a4de5eac"}
Sep 10 21:22:54.284031 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.283964    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6454784d76-w2h5d" podStartSLOduration=2.519903662 podStartE2EDuration="3.283947723s" totalImagesPullingTime="764.044061ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:22:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:22:54.281435452 +0000 UTC m=+750.215102268" watchObservedRunningTime="2026-09-10 21:22:54.283947723 +0000 UTC m=+750.217614538"
Sep 10 21:22:54.313490 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.313451    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c8759f49-lc2rp"]
Sep 10 21:22:54.313722 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.313694    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5c8759f49-lc2rp" podUID="367196de-a050-496d-a13a-bebae099f1a5" containerName="authorino" containerID="cri-o://55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1" gracePeriod=30
Sep 10 21:22:54.554586 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.554554    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c8759f49-lc2rp"
Sep 10 21:22:54.587751 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.587709    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57v6s\" (UniqueName: \"kubernetes.io/projected/367196de-a050-496d-a13a-bebae099f1a5-kube-api-access-57v6s\") pod \"367196de-a050-496d-a13a-bebae099f1a5\" (UID: \"367196de-a050-496d-a13a-bebae099f1a5\") "
Sep 10 21:22:54.587932 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.587761    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/367196de-a050-496d-a13a-bebae099f1a5-tls-cert\") pod \"367196de-a050-496d-a13a-bebae099f1a5\" (UID: \"367196de-a050-496d-a13a-bebae099f1a5\") "
Sep 10 21:22:54.589890 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.589854    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/367196de-a050-496d-a13a-bebae099f1a5-kube-api-access-57v6s" (OuterVolumeSpecName: "kube-api-access-57v6s") pod "367196de-a050-496d-a13a-bebae099f1a5" (UID: "367196de-a050-496d-a13a-bebae099f1a5"). InnerVolumeSpecName "kube-api-access-57v6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:22:54.598800 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.598752    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/367196de-a050-496d-a13a-bebae099f1a5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "367196de-a050-496d-a13a-bebae099f1a5" (UID: "367196de-a050-496d-a13a-bebae099f1a5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:22:54.688568 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.688539    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57v6s\" (UniqueName: \"kubernetes.io/projected/367196de-a050-496d-a13a-bebae099f1a5-kube-api-access-57v6s\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:22:54.688669 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:54.688572    2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/367196de-a050-496d-a13a-bebae099f1a5-tls-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:22:55.271988 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:55.271937    2573 generic.go:356] "Generic (PLEG): container finished" podID="367196de-a050-496d-a13a-bebae099f1a5" containerID="55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1" exitCode=0
Sep 10 21:22:55.271988 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:55.271981    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c8759f49-lc2rp"
Sep 10 21:22:55.272517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:55.272019    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c8759f49-lc2rp" event={"ID":"367196de-a050-496d-a13a-bebae099f1a5","Type":"ContainerDied","Data":"55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1"}
Sep 10 21:22:55.272517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:55.272064    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c8759f49-lc2rp" event={"ID":"367196de-a050-496d-a13a-bebae099f1a5","Type":"ContainerDied","Data":"c9c3734ebf09d8a53742fcf35ef877103ad22ca2a2cc68f8c64a7957e37f3b59"}
Sep 10 21:22:55.272517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:55.272084    2573 scope.go:117] "RemoveContainer" containerID="55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1"
Sep 10 21:22:55.280911 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:55.280895    2573 scope.go:117] "RemoveContainer" containerID="55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1"
Sep 10 21:22:55.281184 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:22:55.281166    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1\": container with ID starting with 55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1 not found: ID does not exist" containerID="55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1"
Sep 10 21:22:55.281236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:55.281196    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1"} err="failed to get container status \"55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1\": rpc error: code = NotFound desc = could not find container \"55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1\": container with ID starting with 55af39f3be2063cac60ef1237452339bb92c5aa8ada84aca9b9f4ec1ae51d6b1 not found: ID does not exist"
Sep 10 21:22:55.312115 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:55.312091    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c8759f49-lc2rp"]
Sep 10 21:22:55.326534 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:55.326503    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5c8759f49-lc2rp"]
Sep 10 21:22:56.679707 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:22:56.679669    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="367196de-a050-496d-a13a-bebae099f1a5" path="/var/lib/kubelet/pods/367196de-a050-496d-a13a-bebae099f1a5/volumes"
Sep 10 21:23:13.208522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.208484    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7803-877c1c"]
Sep 10 21:23:13.208947 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.208847    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="367196de-a050-496d-a13a-bebae099f1a5" containerName="authorino"
Sep 10 21:23:13.208947 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.208861    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="367196de-a050-496d-a13a-bebae099f1a5" containerName="authorino"
Sep 10 21:23:13.208947 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.208931    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="367196de-a050-496d-a13a-bebae099f1a5" containerName="authorino"
Sep 10 21:23:13.211822 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.211806    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7803-877c1c"
Sep 10 21:23:13.214039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.214020    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dxcl2\""
Sep 10 21:23:13.217217 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.217197    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7803-877c1c"]
Sep 10 21:23:13.251093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.251061    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h84nh\" (UniqueName: \"kubernetes.io/projected/ec9172ef-2f99-45f5-b529-ba91dd1476b9-kube-api-access-h84nh\") pod \"test-curl-7803-877c1c\" (UID: \"ec9172ef-2f99-45f5-b529-ba91dd1476b9\") " pod="opendatahub/test-curl-7803-877c1c"
Sep 10 21:23:13.352426 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.352392    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h84nh\" (UniqueName: \"kubernetes.io/projected/ec9172ef-2f99-45f5-b529-ba91dd1476b9-kube-api-access-h84nh\") pod \"test-curl-7803-877c1c\" (UID: \"ec9172ef-2f99-45f5-b529-ba91dd1476b9\") " pod="opendatahub/test-curl-7803-877c1c"
Sep 10 21:23:13.361319 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.361286    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h84nh\" (UniqueName: \"kubernetes.io/projected/ec9172ef-2f99-45f5-b529-ba91dd1476b9-kube-api-access-h84nh\") pod \"test-curl-7803-877c1c\" (UID: \"ec9172ef-2f99-45f5-b529-ba91dd1476b9\") " pod="opendatahub/test-curl-7803-877c1c"
Sep 10 21:23:13.522065 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.521960    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7803-877c1c"
Sep 10 21:23:13.791008 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:13.790981    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7803-877c1c"]
Sep 10 21:23:13.792936 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:23:13.792908    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec9172ef_2f99_45f5_b529_ba91dd1476b9.slice/crio-9a821fb79c73fcf71a9281c44636aa4181b12943a65040750d3be1442d1499dc WatchSource:0}: Error finding container 9a821fb79c73fcf71a9281c44636aa4181b12943a65040750d3be1442d1499dc: Status 404 returned error can't find the container with id 9a821fb79c73fcf71a9281c44636aa4181b12943a65040750d3be1442d1499dc
Sep 10 21:23:14.372388 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:14.372350    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7803-877c1c" event={"ID":"ec9172ef-2f99-45f5-b529-ba91dd1476b9","Type":"ContainerStarted","Data":"9a821fb79c73fcf71a9281c44636aa4181b12943a65040750d3be1442d1499dc"}
Sep 10 21:23:15.120724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:15.120657    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:23:16.386342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:16.386305    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7803-877c1c" event={"ID":"ec9172ef-2f99-45f5-b529-ba91dd1476b9","Type":"ContainerStarted","Data":"1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de"}
Sep 10 21:23:16.400407 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:16.400355    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7803-877c1c" podStartSLOduration=1.365508264 podStartE2EDuration="3.400338373s" totalImagesPullingTime="2.034830109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:16.399535401 +0000 UTC m=+772.333202216" watchObservedRunningTime="2026-09-10 21:23:16.400338373 +0000 UTC m=+772.334005187"
Sep 10 21:23:16.979362 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:16.979325    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7803-877c1c"]
Sep 10 21:23:16.984244 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:16.984214    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7803-877c1c"]
Sep 10 21:23:17.100499 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.100466    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7803-6d3182"]
Sep 10 21:23:17.100857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.100843    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec9172ef-2f99-45f5-b529-ba91dd1476b9" containerName="test-curl-7803-877c1c"
Sep 10 21:23:17.100905 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.100859    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec9172ef-2f99-45f5-b529-ba91dd1476b9" containerName="test-curl-7803-877c1c"
Sep 10 21:23:17.100941 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.100923    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec9172ef-2f99-45f5-b529-ba91dd1476b9" containerName="test-curl-7803-877c1c"
Sep 10 21:23:17.103127 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.103110    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:17.112946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.112920    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7803-6d3182"]
Sep 10 21:23:17.183513 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.183475    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zlfq\" (UniqueName: \"kubernetes.io/projected/efacf81e-681d-43b3-815c-b14cc9896634-kube-api-access-5zlfq\") pod \"test-curl-7803-6d3182\" (UID: \"efacf81e-681d-43b3-815c-b14cc9896634\") " pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:17.284436 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.284400    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zlfq\" (UniqueName: \"kubernetes.io/projected/efacf81e-681d-43b3-815c-b14cc9896634-kube-api-access-5zlfq\") pod \"test-curl-7803-6d3182\" (UID: \"efacf81e-681d-43b3-815c-b14cc9896634\") " pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:17.292828 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.292769    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zlfq\" (UniqueName: \"kubernetes.io/projected/efacf81e-681d-43b3-815c-b14cc9896634-kube-api-access-5zlfq\") pod \"test-curl-7803-6d3182\" (UID: \"efacf81e-681d-43b3-815c-b14cc9896634\") " pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:17.391889 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:17.391810    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7803-877c1c" container="test-curl-7803-877c1c"
Sep 10 21:23:17.412762 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.412735    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:17.682004 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:17.681980    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7803-6d3182"]
Sep 10 21:23:17.684057 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:23:17.684031    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefacf81e_681d_43b3_815c_b14cc9896634.slice/crio-39e317578ef046a428bf71543b19ab4b890af966a471b5cbb3fe0315d2d3ac84 WatchSource:0}: Error finding container 39e317578ef046a428bf71543b19ab4b890af966a471b5cbb3fe0315d2d3ac84: Status 404 returned error can't find the container with id 39e317578ef046a428bf71543b19ab4b890af966a471b5cbb3fe0315d2d3ac84
Sep 10 21:23:18.398005 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:18.397963    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7803-6d3182" event={"ID":"efacf81e-681d-43b3-815c-b14cc9896634","Type":"ContainerStarted","Data":"149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c"}
Sep 10 21:23:18.398478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:18.398016    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7803-6d3182" event={"ID":"efacf81e-681d-43b3-815c-b14cc9896634","Type":"ContainerStarted","Data":"39e317578ef046a428bf71543b19ab4b890af966a471b5cbb3fe0315d2d3ac84"}
Sep 10 21:23:18.398478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:18.398060    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7803-877c1c" podUID="ec9172ef-2f99-45f5-b529-ba91dd1476b9" containerName="test-curl-7803-877c1c" containerID="cri-o://1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de" gracePeriod=2
Sep 10 21:23:18.399691 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:18.399667    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7803-877c1c\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="ec9172ef-2f99-45f5-b529-ba91dd1476b9" pod="opendatahub/test-curl-7803-877c1c"
Sep 10 21:23:18.413181 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:18.413129    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7803-6d3182" podStartSLOduration=1.413112152 podStartE2EDuration="1.413112152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:18.412331934 +0000 UTC m=+774.345998761" watchObservedRunningTime="2026-09-10 21:23:18.413112152 +0000 UTC m=+774.346778981"
Sep 10 21:23:18.414029 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:18.414004    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7803-877c1c\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="ec9172ef-2f99-45f5-b529-ba91dd1476b9" pod="opendatahub/test-curl-7803-877c1c"
Sep 10 21:23:19.005959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.005926    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7803-6d3182"]
Sep 10 21:23:19.010569 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.010539    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7803-6d3182"]
Sep 10 21:23:19.357612 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.357581    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"]
Sep 10 21:23:19.357989 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.357975    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efacf81e-681d-43b3-815c-b14cc9896634" containerName="test-curl-7803-6d3182"
Sep 10 21:23:19.358049 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.357991    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="efacf81e-681d-43b3-815c-b14cc9896634" containerName="test-curl-7803-6d3182"
Sep 10 21:23:19.358129 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.358064    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="efacf81e-681d-43b3-815c-b14cc9896634" containerName="test-curl-7803-6d3182"
Sep 10 21:23:19.360280 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.360264    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.362501 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.362478    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 21:23:19.362617 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.362573    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-780a0cdd-openshift-default-dockercfg-qj5gp\""
Sep 10 21:23:19.374122 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.374100    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"]
Sep 10 21:23:19.402686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.402652    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-socket\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.403069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.402700    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-envoy\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.403069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.402734    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-token\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.403069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.402752    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-credential-socket\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.403069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.402807    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istiod-ca-cert\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.403069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.402856    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-certs\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.403069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.402881    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-data\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.403069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.402896    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-podinfo\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.403069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.402915    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6x82\" (UniqueName: \"kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-kube-api-access-l6x82\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.403907 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:19.403888    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7803-6d3182" container="test-curl-7803-6d3182"
Sep 10 21:23:19.503979 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.503942    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-data\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.503979 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.503977    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-podinfo\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504185 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.503997    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6x82\" (UniqueName: \"kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-kube-api-access-l6x82\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504185 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504026    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-socket\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504185 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504074    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-envoy\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504185 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504106    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-token\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504185 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504132    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-credential-socket\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504185 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504183    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istiod-ca-cert\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504450 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504216    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-certs\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504450 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504414    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-data\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504450 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504427    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-socket\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504735    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-certs\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504912 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504889    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-credential-socket\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.504986 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.504966    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istiod-ca-cert\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.506570 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.506549    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-envoy\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.506657 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.506581    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-podinfo\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.512197 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.512175    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-token\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.512468 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.512442    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6x82\" (UniqueName: \"kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-kube-api-access-l6x82\") pod \"e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") " pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.670182 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.670087    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:19.821033 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.821003    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"]
Sep 10 21:23:19.823584 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.823567    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:19.828625 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.828601    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-f135f4-openshift-default-dockercfg-tcthb\""
Sep 10 21:23:19.850157 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.850138    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"]
Sep 10 21:23:19.908787 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.908750    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-podinfo\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:19.908961 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.908813    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-credential-socket\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:19.908961 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.908837    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-data\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:19.908961 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.908856    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac73d580-91be-4437-8103-1b1e70b0fc95-istiod-ca-cert\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:19.908961 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.908890    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdn5w\" (UniqueName: \"kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-kube-api-access-fdn5w\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:19.909113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.909005    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-envoy\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:19.909113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.909037    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-socket\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:19.909113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.909057    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-token\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:19.909113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.909079    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-certs\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:19.946466 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:19.946439    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"]
Sep 10 21:23:19.947686 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:23:19.947660    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ae7fad_c358_48f8_ac24_e3d236a6b7c5.slice/crio-17ac2b2f49b99181144e7ca459419f1d829a8966844744439f779acc87a1779a WatchSource:0}: Error finding container 17ac2b2f49b99181144e7ca459419f1d829a8966844744439f779acc87a1779a: Status 404 returned error can't find the container with id 17ac2b2f49b99181144e7ca459419f1d829a8966844744439f779acc87a1779a
Sep 10 21:23:20.009845 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.009808    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-podinfo\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010026 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.009878    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-credential-socket\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010026 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.009908    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-data\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010026 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.009943    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac73d580-91be-4437-8103-1b1e70b0fc95-istiod-ca-cert\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010026 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.010007    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdn5w\" (UniqueName: \"kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-kube-api-access-fdn5w\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.010100    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-envoy\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.010128    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-socket\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.010157    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-token\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.010185    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-certs\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010452 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.010246    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-credential-socket\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010452 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.010307    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-data\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010557 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.010528    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-certs\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010622 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.010558    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-socket\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.010763 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.010742    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac73d580-91be-4437-8103-1b1e70b0fc95-istiod-ca-cert\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.012376 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.012354    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-envoy\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.012448 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.012372    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-podinfo\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.019326 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.019294    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-token\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.019550 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.019530    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdn5w\" (UniqueName: \"kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-kube-api-access-fdn5w\") pod \"e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") " pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.135437 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.135395    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:20.407623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.407592    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"]
Sep 10 21:23:20.412912 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.412840    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" event={"ID":"31ae7fad-c358-48f8-ac24-e3d236a6b7c5","Type":"ContainerStarted","Data":"17ac2b2f49b99181144e7ca459419f1d829a8966844744439f779acc87a1779a"}
Sep 10 21:23:20.413015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.412981    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7803-6d3182" podUID="efacf81e-681d-43b3-815c-b14cc9896634" containerName="test-curl-7803-6d3182" containerID="cri-o://149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c" gracePeriod=2
Sep 10 21:23:20.415101 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:20.415076    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7803-6d3182\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="efacf81e-681d-43b3-815c-b14cc9896634" pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:20.454936 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:23:20.454889    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac73d580_91be_4437_8103_1b1e70b0fc95.slice/crio-8c4c94fc10d7d1407b8959dcc271109285855e2dcf411232d342f8fb1013083d WatchSource:0}: Error finding container 8c4c94fc10d7d1407b8959dcc271109285855e2dcf411232d342f8fb1013083d: Status 404 returned error can't find the container with id 8c4c94fc10d7d1407b8959dcc271109285855e2dcf411232d342f8fb1013083d
Sep 10 21:23:20.543725 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.543699    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7803-877c1c"
Sep 10 21:23:20.545747 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:20.545718    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7803-6d3182\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="efacf81e-681d-43b3-815c-b14cc9896634" pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:20.616103 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.616072    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h84nh\" (UniqueName: \"kubernetes.io/projected/ec9172ef-2f99-45f5-b529-ba91dd1476b9-kube-api-access-h84nh\") pod \"ec9172ef-2f99-45f5-b529-ba91dd1476b9\" (UID: \"ec9172ef-2f99-45f5-b529-ba91dd1476b9\") "
Sep 10 21:23:20.618230 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.618196    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec9172ef-2f99-45f5-b529-ba91dd1476b9-kube-api-access-h84nh" (OuterVolumeSpecName: "kube-api-access-h84nh") pod "ec9172ef-2f99-45f5-b529-ba91dd1476b9" (UID: "ec9172ef-2f99-45f5-b529-ba91dd1476b9"). InnerVolumeSpecName "kube-api-access-h84nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:23:20.679866 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.679789    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec9172ef-2f99-45f5-b529-ba91dd1476b9" path="/var/lib/kubelet/pods/ec9172ef-2f99-45f5-b529-ba91dd1476b9/volumes"
Sep 10 21:23:20.716962 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:20.716932    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h84nh\" (UniqueName: \"kubernetes.io/projected/ec9172ef-2f99-45f5-b529-ba91dd1476b9-kube-api-access-h84nh\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:23:21.422358 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:21.422313    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" event={"ID":"ac73d580-91be-4437-8103-1b1e70b0fc95","Type":"ContainerStarted","Data":"8c4c94fc10d7d1407b8959dcc271109285855e2dcf411232d342f8fb1013083d"}
Sep 10 21:23:21.424229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:21.423985    2573 generic.go:356] "Generic (PLEG): container finished" podID="ec9172ef-2f99-45f5-b529-ba91dd1476b9" containerID="1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de" exitCode=137
Sep 10 21:23:21.424229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:21.424097    2573 scope.go:117] "RemoveContainer" containerID="1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de"
Sep 10 21:23:21.424229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:21.424192    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7803-877c1c"
Sep 10 21:23:21.436931 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:21.436907    2573 scope.go:117] "RemoveContainer" containerID="1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de"
Sep 10 21:23:21.437445 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:21.437290    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de\": container with ID starting with 1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de not found: ID does not exist" containerID="1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de"
Sep 10 21:23:21.437445 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:21.437335    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de"} err="failed to get container status \"1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de\": rpc error: code = NotFound desc = could not find container \"1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de\": container with ID starting with 1dfd4425bca339d6ea049962b2d04dacd0b0a5d752afa0b4d2f4244c099cf5de not found: ID does not exist"
Sep 10 21:23:22.935271 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:22.935230    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:22.935619 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:22.935305    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:22.935619 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:22.935334    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:22.941122 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:22.941095    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:22.941213 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:22.941154    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:22.941213 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:22.941181    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:23.039395 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.039368    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:23.138318 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.138282    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zlfq\" (UniqueName: \"kubernetes.io/projected/efacf81e-681d-43b3-815c-b14cc9896634-kube-api-access-5zlfq\") pod \"efacf81e-681d-43b3-815c-b14cc9896634\" (UID: \"efacf81e-681d-43b3-815c-b14cc9896634\") "
Sep 10 21:23:23.140816 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.140757    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efacf81e-681d-43b3-815c-b14cc9896634-kube-api-access-5zlfq" (OuterVolumeSpecName: "kube-api-access-5zlfq") pod "efacf81e-681d-43b3-815c-b14cc9896634" (UID: "efacf81e-681d-43b3-815c-b14cc9896634"). InnerVolumeSpecName "kube-api-access-5zlfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:23:23.240233 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.240146    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zlfq\" (UniqueName: \"kubernetes.io/projected/efacf81e-681d-43b3-815c-b14cc9896634-kube-api-access-5zlfq\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:23:23.437683 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.437651    2573 generic.go:356] "Generic (PLEG): container finished" podID="efacf81e-681d-43b3-815c-b14cc9896634" containerID="149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c" exitCode=137
Sep 10 21:23:23.437912 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.437700    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:23.437912 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.437753    2573 scope.go:117] "RemoveContainer" containerID="149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c"
Sep 10 21:23:23.439610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.439582    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" event={"ID":"ac73d580-91be-4437-8103-1b1e70b0fc95","Type":"ContainerStarted","Data":"9bbfe76e3c2224bb9d7957703ac58f9ac36c27fb8f5b7a0d711649c9421ac7f5"}
Sep 10 21:23:23.441440 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.441415    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" event={"ID":"31ae7fad-c358-48f8-ac24-e3d236a6b7c5","Type":"ContainerStarted","Data":"90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d"}
Sep 10 21:23:23.448136 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.447914    2573 scope.go:117] "RemoveContainer" containerID="149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c"
Sep 10 21:23:23.448441 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:23.448205    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c\": container with ID starting with 149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c not found: ID does not exist" containerID="149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c"
Sep 10 21:23:23.448441 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.448241    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c"} err="failed to get container status \"149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c\": rpc error: code = NotFound desc = could not find container \"149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c\": container with ID starting with 149c3932622f237129ae316c510ce7bc6b0f095077aa6f1f98044cff3a7fd77c not found: ID does not exist"
Sep 10 21:23:23.461980 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.461930    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" podStartSLOduration=1.978108652 podStartE2EDuration="4.461914076s" totalImagesPullingTime="2.483805424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:23.460140482 +0000 UTC m=+779.393807298" watchObservedRunningTime="2026-09-10 21:23:23.461914076 +0000 UTC m=+779.395580891"
Sep 10 21:23:23.482459 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:23.482419    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7803-6d3182\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="efacf81e-681d-43b3-815c-b14cc9896634" pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:23.483955 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.483904    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" podStartSLOduration=1.498807781 podStartE2EDuration="4.483887558s" totalImagesPullingTime="2.985079777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:23.480626802 +0000 UTC m=+779.414293620" watchObservedRunningTime="2026-09-10 21:23:23.483887558 +0000 UTC m=+779.417554372"
Sep 10 21:23:23.670727 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:23.670696    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:24.135919 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:24.135881    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:24.140839 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:24.140814    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:24.148466 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:24.148435    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7803-6d3182\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="efacf81e-681d-43b3-815c-b14cc9896634" pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:24.449627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:24.449539    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:24.450568 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:24.450544    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:24.452616 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:24.452583    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7803-6d3182\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="efacf81e-681d-43b3-815c-b14cc9896634" pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:24.678435 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:24.678401    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7803-6d3182\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="efacf81e-681d-43b3-815c-b14cc9896634" pod="opendatahub/test-curl-7803-6d3182"
Sep 10 21:23:24.679879 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:24.679851    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efacf81e-681d-43b3-815c-b14cc9896634" path="/var/lib/kubelet/pods/efacf81e-681d-43b3-815c-b14cc9896634/volumes"
Sep 10 21:23:24.680223 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:24.680206    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:25.454299 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:25.454264    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:25.455310 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:25.455291    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:29.713952 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:29.713910    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:23:39.734336 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.734301    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"]
Sep 10 21:23:39.737322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.737302    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.739767 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.739735    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a6441b-openshift-default-dockercfg-q5k59\""
Sep 10 21:23:39.750746 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.750721    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"]
Sep 10 21:23:39.790343 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.790304    2573 patch_prober.go:28] interesting pod/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:43854->10.134.0.45:15021: read: connection reset by peer" start-of-body=
Sep 10 21:23:39.790485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.790399    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:43854->10.134.0.45:15021: read: connection reset by peer"
Sep 10 21:23:39.892379 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.892343    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-socket\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.892543 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.892393    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-data\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.892543 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.892412    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c547695c-7b06-419c-9de4-dec615374497-istiod-ca-cert\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.892543 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.892493    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-envoy\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.892543 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.892516    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c547695c-7b06-419c-9de4-dec615374497-istio-podinfo\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.892687 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.892550    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-credential-socket\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.892687 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.892613    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsmzn\" (UniqueName: \"kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-kube-api-access-qsmzn\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.892687 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.892668    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-certs\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.892815 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.892705    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-istio-token\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994058 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.993955    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-credential-socket\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994058 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994005    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsmzn\" (UniqueName: \"kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-kube-api-access-qsmzn\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994058 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994039    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-certs\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994071    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-istio-token\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994105    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-socket\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994149    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-data\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994523 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994309    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c547695c-7b06-419c-9de4-dec615374497-istiod-ca-cert\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994523 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994358    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-envoy\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994523 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994380    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c547695c-7b06-419c-9de4-dec615374497-istio-podinfo\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994523 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994407    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-credential-socket\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994523 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994436    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-certs\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994722 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994545    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-socket\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.994722 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.994610    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-data\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.995052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.995025    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c547695c-7b06-419c-9de4-dec615374497-istiod-ca-cert\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.996588 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.996557    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-envoy\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:39.996840 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:39.996821    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c547695c-7b06-419c-9de4-dec615374497-istio-podinfo\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:40.003735 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.003692    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsmzn\" (UniqueName: \"kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-kube-api-access-qsmzn\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:40.003972 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.003949    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-istio-token\") pod \"e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") " pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:40.052990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.052960    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:40.323823 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.323797    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"]
Sep 10 21:23:40.324932 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:23:40.324895    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc547695c_7b06_419c_9de4_dec615374497.slice/crio-3b6e0475270866477ab9aea72313c1a42ad343e8da208249a7e2013fa7e7f0a1 WatchSource:0}: Error finding container 3b6e0475270866477ab9aea72313c1a42ad343e8da208249a7e2013fa7e7f0a1: Status 404 returned error can't find the container with id 3b6e0475270866477ab9aea72313c1a42ad343e8da208249a7e2013fa7e7f0a1
Sep 10 21:23:40.327127 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.327096    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:40.327190 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.327169    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:40.327229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.327195    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:40.455162 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.455125    2573 patch_prober.go:28] interesting pod/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 10 21:23:40.455348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.455180    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 10 21:23:40.552662 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.552631    2573 generic.go:356] "Generic (PLEG): container finished" podID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerID="9bbfe76e3c2224bb9d7957703ac58f9ac36c27fb8f5b7a0d711649c9421ac7f5" exitCode=0
Sep 10 21:23:40.552862 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.552717    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" event={"ID":"ac73d580-91be-4437-8103-1b1e70b0fc95","Type":"ContainerDied","Data":"9bbfe76e3c2224bb9d7957703ac58f9ac36c27fb8f5b7a0d711649c9421ac7f5"}
Sep 10 21:23:40.553290 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.553255    2573 scope.go:117] "RemoveContainer" containerID="9bbfe76e3c2224bb9d7957703ac58f9ac36c27fb8f5b7a0d711649c9421ac7f5"
Sep 10 21:23:40.554489 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.554444    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:40.554582 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.554534    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:40.554582 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.554555    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" event={"ID":"31ae7fad-c358-48f8-ac24-e3d236a6b7c5","Type":"ContainerDied","Data":"90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d"}
Sep 10 21:23:40.554660 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.554584    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:40.554730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.554536    2573 generic.go:356] "Generic (PLEG): container finished" podID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerID="90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d" exitCode=0
Sep 10 21:23:40.554921 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.554907    2573 scope.go:117] "RemoveContainer" containerID="90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d"
Sep 10 21:23:40.556031 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.555993    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:40.556239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.556071    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:40.556239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.556118    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:23:40.556847 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.556819    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" event={"ID":"c547695c-7b06-419c-9de4-dec615374497","Type":"ContainerStarted","Data":"b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904"}
Sep 10 21:23:40.556946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.556874    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" event={"ID":"c547695c-7b06-419c-9de4-dec615374497","Type":"ContainerStarted","Data":"3b6e0475270866477ab9aea72313c1a42ad343e8da208249a7e2013fa7e7f0a1"}
Sep 10 21:23:40.691241 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:40.691181    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" podStartSLOduration=1.6911588800000001 podStartE2EDuration="1.69115888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:40.645516652 +0000 UTC m=+796.579183471" watchObservedRunningTime="2026-09-10 21:23:40.69115888 +0000 UTC m=+796.624825689"
Sep 10 21:23:41.053968 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:41.053909    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:41.060861 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:41.060823    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:41.564336 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:41.564297    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" event={"ID":"ac73d580-91be-4437-8103-1b1e70b0fc95","Type":"ContainerStarted","Data":"235c13c77e83491ebdb54478bf705848e38d4039b4ea5847b0e6ef6073b4cfb3"}
Sep 10 21:23:41.566594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:41.566566    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" event={"ID":"31ae7fad-c358-48f8-ac24-e3d236a6b7c5","Type":"ContainerStarted","Data":"fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54"}
Sep 10 21:23:41.566896 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:41.566876    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:41.567699 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:41.567682    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:23:41.670483 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:41.670451    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:41.670651 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:41.670559    2573 patch_prober.go:28] interesting pod/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 10 21:23:41.670651 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:41.670606    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 10 21:23:42.136135 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:42.136091    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:42.136514 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:42.136297    2573 patch_prober.go:28] interesting pod/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 21:23:42.136514 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:42.136344    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 21:23:42.670572 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:42.670535    2573 patch_prober.go:28] interesting pod/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 10 21:23:42.670811 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:42.670594    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 10 21:23:43.136837 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.136770    2573 patch_prober.go:28] interesting pod/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 21:23:43.137335 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.136878    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 21:23:43.671192 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.671152    2573 patch_prober.go:28] interesting pod/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 10 21:23:43.671500 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.671469    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 10 21:23:43.718946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.718904    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"]
Sep 10 21:23:43.729482 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.729440    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.732545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.732514    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a6441b-serving-cert\""
Sep 10 21:23:43.732762 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.732737    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:23:43.733358 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.733117    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:23:43.733615 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.733170    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:23:43.733874 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.733262    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-99rzw\""
Sep 10 21:23:43.734331 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.734310    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"]
Sep 10 21:23:43.837654 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.837620    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4pz7\" (UniqueName: \"kubernetes.io/projected/5dd07a8c-df59-480d-982c-f69adc174e72-kube-api-access-w4pz7\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.837837 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.837818    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd07a8c-df59-480d-982c-f69adc174e72-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.837913 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.837848    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-metrics-certs\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.837913 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.837881    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.938602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.938511    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4pz7\" (UniqueName: \"kubernetes.io/projected/5dd07a8c-df59-480d-982c-f69adc174e72-kube-api-access-w4pz7\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.938602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.938601    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd07a8c-df59-480d-982c-f69adc174e72-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.938860 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.938618    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-metrics-certs\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.938860 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.938640    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.938860 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:43.938747    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-serving-cert: secret "maas-api-e2e-inf-b-a6441b-serving-cert" not found
Sep 10 21:23:43.938860 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:43.938809    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls podName:5dd07a8c-df59-480d-982c-f69adc174e72 nodeName:}" failed. No retries permitted until 2026-09-10 21:23:44.438793698 +0000 UTC m=+800.372460491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls") pod "maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l" (UID: "5dd07a8c-df59-480d-982c-f69adc174e72") : secret "maas-api-e2e-inf-b-a6441b-serving-cert" not found
Sep 10 21:23:43.939652 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.939628    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd07a8c-df59-480d-982c-f69adc174e72-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.941136 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.941104    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-metrics-certs\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:43.946567 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:43.946540    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4pz7\" (UniqueName: \"kubernetes.io/projected/5dd07a8c-df59-480d-982c-f69adc174e72-kube-api-access-w4pz7\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:44.136065 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:44.136022    2573 patch_prober.go:28] interesting pod/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 21:23:44.136256 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:44.136083    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 21:23:44.430484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:44.430444    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:23:44.443498 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:44.443473    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:44.443768 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:44.443748    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-serving-cert: secret "maas-api-e2e-inf-b-a6441b-serving-cert" not found
Sep 10 21:23:44.443853 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:23:44.443833    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls podName:5dd07a8c-df59-480d-982c-f69adc174e72 nodeName:}" failed. No retries permitted until 2026-09-10 21:23:45.443818375 +0000 UTC m=+801.377485168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls") pod "maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l" (UID: "5dd07a8c-df59-480d-982c-f69adc174e72") : secret "maas-api-e2e-inf-b-a6441b-serving-cert" not found
Sep 10 21:23:44.674803 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:44.671209    2573 patch_prober.go:28] interesting pod/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 10 21:23:44.674803 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:44.671273    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 10 21:23:45.136147 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:45.136108    2573 patch_prober.go:28] interesting pod/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 21:23:45.136327 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:45.136168    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 21:23:45.455104 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:45.454995    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:45.457545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:45.457523    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:45.543914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:45.543883    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:45.670455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:45.670424    2573 patch_prober.go:28] interesting pod/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 10 21:23:45.670613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:45.670477    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 10 21:23:45.867580 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:45.867543    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"]
Sep 10 21:23:45.868199 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:23:45.868173    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd07a8c_df59_480d_982c_f69adc174e72.slice/crio-e4db5adbc966abc2bd4cc232840dceeb86b4de0b45c4a0b6e33919b700355c14 WatchSource:0}: Error finding container e4db5adbc966abc2bd4cc232840dceeb86b4de0b45c4a0b6e33919b700355c14: Status 404 returned error can't find the container with id e4db5adbc966abc2bd4cc232840dceeb86b4de0b45c4a0b6e33919b700355c14
Sep 10 21:23:46.136599 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:46.136515    2573 patch_prober.go:28] interesting pod/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 21:23:46.136599 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:46.136578    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 21:23:46.632018 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:46.631942    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l" event={"ID":"5dd07a8c-df59-480d-982c-f69adc174e72","Type":"ContainerStarted","Data":"e4db5adbc966abc2bd4cc232840dceeb86b4de0b45c4a0b6e33919b700355c14"}
Sep 10 21:23:46.675463 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:46.670695    2573 patch_prober.go:28] interesting pod/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 10 21:23:46.675463 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:46.670762    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 10 21:23:47.140700 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:47.140667    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:47.141123 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:47.141089    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:47.141873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:47.141849    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:23:47.675748 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:47.675705    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:47.676259 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:47.676233    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:47.677177 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:47.677152    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:23:48.645841 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:48.645799    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l" event={"ID":"5dd07a8c-df59-480d-982c-f69adc174e72","Type":"ContainerStarted","Data":"e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af"}
Sep 10 21:23:48.646034 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:48.645896    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:48.666901 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:48.666852    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l" podStartSLOduration=3.236326082 podStartE2EDuration="5.666833802s" totalImagesPullingTime="2.43050772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:48.664712703 +0000 UTC m=+804.598379511" watchObservedRunningTime="2026-09-10 21:23:48.666833802 +0000 UTC m=+804.600500616"
Sep 10 21:23:50.296351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:50.296312    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"]
Sep 10 21:23:50.299107 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:50.299088    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"
Sep 10 21:23:50.301099 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:50.301078    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:23:50.307593 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:50.307568    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"]
Sep 10 21:23:50.398429 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:50.398395    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/976fe0e5-d59c-48d5-abdb-96d9d2210532-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-780a0cdd-rt768\" (UID: \"976fe0e5-d59c-48d5-abdb-96d9d2210532\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"
Sep 10 21:23:50.499485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:50.499446    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/976fe0e5-d59c-48d5-abdb-96d9d2210532-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-780a0cdd-rt768\" (UID: \"976fe0e5-d59c-48d5-abdb-96d9d2210532\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"
Sep 10 21:23:50.500145 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:50.500123    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/976fe0e5-d59c-48d5-abdb-96d9d2210532-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-780a0cdd-rt768\" (UID: \"976fe0e5-d59c-48d5-abdb-96d9d2210532\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"
Sep 10 21:23:50.610700 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:50.610661    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"
Sep 10 21:23:50.884602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:50.884569    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"]
Sep 10 21:23:50.885662 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:23:50.885629    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod976fe0e5_d59c_48d5_abdb_96d9d2210532.slice/crio-347c94757d9018950528d339640ec361b4bfd82a665051589a3740fb94579043 WatchSource:0}: Error finding container 347c94757d9018950528d339640ec361b4bfd82a665051589a3740fb94579043: Status 404 returned error can't find the container with id 347c94757d9018950528d339640ec361b4bfd82a665051589a3740fb94579043
Sep 10 21:23:51.662529 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:51.662496    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768" event={"ID":"976fe0e5-d59c-48d5-abdb-96d9d2210532","Type":"ContainerStarted","Data":"347c94757d9018950528d339640ec361b4bfd82a665051589a3740fb94579043"}
Sep 10 21:23:54.656818 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:54.656733    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:23:54.692953 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:54.692917    2573 generic.go:356] "Generic (PLEG): container finished" podID="976fe0e5-d59c-48d5-abdb-96d9d2210532" containerID="64db6ace0722856d32e664cdb6db63e8f21c45cd4f3563426faf52c63e86d9fc" exitCode=0
Sep 10 21:23:54.701802 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:54.701663    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768" event={"ID":"976fe0e5-d59c-48d5-abdb-96d9d2210532","Type":"ContainerDied","Data":"64db6ace0722856d32e664cdb6db63e8f21c45cd4f3563426faf52c63e86d9fc"}
Sep 10 21:23:55.852501 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:55.852474    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"
Sep 10 21:23:55.956972 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:55.956864    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/976fe0e5-d59c-48d5-abdb-96d9d2210532-maas-api-service-ca\") pod \"976fe0e5-d59c-48d5-abdb-96d9d2210532\" (UID: \"976fe0e5-d59c-48d5-abdb-96d9d2210532\") "
Sep 10 21:23:55.957221 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:55.957198    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/976fe0e5-d59c-48d5-abdb-96d9d2210532-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "976fe0e5-d59c-48d5-abdb-96d9d2210532" (UID: "976fe0e5-d59c-48d5-abdb-96d9d2210532"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:23:56.057977 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:56.057944    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/976fe0e5-d59c-48d5-abdb-96d9d2210532-maas-api-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:23:56.706322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:56.706292    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"
Sep 10 21:23:56.706322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:56.706305    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768" event={"ID":"976fe0e5-d59c-48d5-abdb-96d9d2210532","Type":"ContainerDied","Data":"347c94757d9018950528d339640ec361b4bfd82a665051589a3740fb94579043"}
Sep 10 21:23:56.706511 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:56.706350    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="347c94757d9018950528d339640ec361b4bfd82a665051589a3740fb94579043"
Sep 10 21:23:57.310393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:23:57.310355    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:24:00.737349 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:00.737312    2573 generic.go:356] "Generic (PLEG): container finished" podID="c547695c-7b06-419c-9de4-dec615374497" containerID="b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904" exitCode=0
Sep 10 21:24:00.737762 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:00.737384    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" event={"ID":"c547695c-7b06-419c-9de4-dec615374497","Type":"ContainerDied","Data":"b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904"}
Sep 10 21:24:00.737853 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:00.737825    2573 scope.go:117] "RemoveContainer" containerID="b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904"
Sep 10 21:24:00.739039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:00.739001    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:00.739158 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:00.739093    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:00.739158 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:00.739146    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:01.745263 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:01.745226    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" event={"ID":"c547695c-7b06-419c-9de4-dec615374497","Type":"ContainerStarted","Data":"a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413"}
Sep 10 21:24:02.053526 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:02.053436    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:24:02.053694 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:02.053674    2573 patch_prober.go:28] interesting pod/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 10 21:24:02.053750 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:02.053732    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 10 21:24:03.053885 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:03.053764    2573 patch_prober.go:28] interesting pod/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 10 21:24:03.053885 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:03.053867    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 10 21:24:04.053544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:04.053499    2573 patch_prober.go:28] interesting pod/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 10 21:24:04.053758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:04.053565    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 10 21:24:05.053740 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:05.053701    2573 patch_prober.go:28] interesting pod/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 10 21:24:05.054121 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:05.053752    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 10 21:24:06.054124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:06.054087    2573 patch_prober.go:28] interesting pod/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 10 21:24:06.054500 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:06.054144    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 10 21:24:06.411692 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:06.411658    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:24:07.053463 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:07.053428    2573 patch_prober.go:28] interesting pod/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 10 21:24:07.053634 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:07.053477    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 10 21:24:07.069292 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:07.069250    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"]
Sep 10 21:24:07.080816 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:07.080787    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-780a0cdd-rt768"]
Sep 10 21:24:07.218298 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:07.218261    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"]
Sep 10 21:24:07.218556 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:07.218506    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy" containerID="cri-o://fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54" gracePeriod=30
Sep 10 21:24:08.058334 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.058305    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:24:08.058731 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.058710    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:24:08.059273 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.059253    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:24:08.182882 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.182852    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"]
Sep 10 21:24:08.183328 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.183218    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="976fe0e5-d59c-48d5-abdb-96d9d2210532" containerName="revoke-keys"
Sep 10 21:24:08.183328 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.183230    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="976fe0e5-d59c-48d5-abdb-96d9d2210532" containerName="revoke-keys"
Sep 10 21:24:08.183328 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.183292    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="976fe0e5-d59c-48d5-abdb-96d9d2210532" containerName="revoke-keys"
Sep 10 21:24:08.185974 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.185955    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.190231 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.190200    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-23d91506-gw-openshift-default-dockercfg-qzm8z\""
Sep 10 21:24:08.199021 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.198997    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"]
Sep 10 21:24:08.273707 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.273669    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-422nq\" (UniqueName: \"kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-kube-api-access-422nq\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.273903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.273712    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-credential-socket\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.273903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.273821    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istiod-ca-cert\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.273903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.273859    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-podinfo\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.273903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.273901    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-token\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.274039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.273923    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-envoy\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.274039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.273958    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-data\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.274039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.274013    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-certs\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.274039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.274031    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-socket\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.374505 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.374469    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istiod-ca-cert\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.374505 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.374515    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-podinfo\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.374745 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.374644    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-token\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.374745 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.374686    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-envoy\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.374745 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.374717    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-data\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.374922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.374752    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-certs\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.374922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.374808    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-socket\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.374922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.374902    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-422nq\" (UniqueName: \"kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-kube-api-access-422nq\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.375088 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.374984    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-credential-socket\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.375180 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.375149    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-data\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.375324 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.375288    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istiod-ca-cert\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.375382 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.375334    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-certs\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.375447 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.375424    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-socket\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.375484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.375445    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-credential-socket\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.377057 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.377040    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-podinfo\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.377124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.377075    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-envoy\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.403971 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.403942    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-token\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.408208 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.408186    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-422nq\" (UniqueName: \"kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-kube-api-access-422nq\") pod \"e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") " pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.496429 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.496391    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:08.679190 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.679111    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="976fe0e5-d59c-48d5-abdb-96d9d2210532" path="/var/lib/kubelet/pods/976fe0e5-d59c-48d5-abdb-96d9d2210532/volumes"
Sep 10 21:24:08.779981 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.779952    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"]
Sep 10 21:24:08.780568 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:24:08.780546    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8a65cd5_aa12_4280_87a3_9a8e96bf6ffa.slice/crio-ca723a0d99a6899b297b9a577fb1515ba0b872f074499fbe47c6f24f9a35b1d9 WatchSource:0}: Error finding container ca723a0d99a6899b297b9a577fb1515ba0b872f074499fbe47c6f24f9a35b1d9: Status 404 returned error can't find the container with id ca723a0d99a6899b297b9a577fb1515ba0b872f074499fbe47c6f24f9a35b1d9
Sep 10 21:24:08.789216 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.783332    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:08.789216 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.783572    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:08.789216 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.783672    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:08.795764 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:08.795724    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" event={"ID":"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa","Type":"ContainerStarted","Data":"ca723a0d99a6899b297b9a577fb1515ba0b872f074499fbe47c6f24f9a35b1d9"}
Sep 10 21:24:09.802822 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:09.802768    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" event={"ID":"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa","Type":"ContainerStarted","Data":"d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5"}
Sep 10 21:24:09.850031 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:09.849979    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" podStartSLOduration=1.849961848 podStartE2EDuration="1.849961848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:09.847027392 +0000 UTC m=+825.780694206" watchObservedRunningTime="2026-09-10 21:24:09.849961848 +0000 UTC m=+825.783628663"
Sep 10 21:24:10.497526 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:10.497489    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:10.502465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:10.502439    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:10.808051 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:10.807962    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:10.808968 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:10.808949    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:12.472592 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.472567    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:24:12.620115 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620067    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-data\") pod \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") "
Sep 10 21:24:12.620115 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620117    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6x82\" (UniqueName: \"kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-kube-api-access-l6x82\") pod \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") "
Sep 10 21:24:12.620385 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620147    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-credential-socket\") pod \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") "
Sep 10 21:24:12.620385 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620198    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-envoy\") pod \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") "
Sep 10 21:24:12.620385 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620220    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-socket\") pod \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") "
Sep 10 21:24:12.620385 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620280    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-token\") pod \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") "
Sep 10 21:24:12.620385 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620344    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-podinfo\") pod \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") "
Sep 10 21:24:12.620385 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620374    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-certs\") pod \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") "
Sep 10 21:24:12.620686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620377    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-data" (OuterVolumeSpecName: "istio-data") pod "31ae7fad-c358-48f8-ac24-e3d236a6b7c5" (UID: "31ae7fad-c358-48f8-ac24-e3d236a6b7c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:24:12.620686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620430    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istiod-ca-cert\") pod \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\" (UID: \"31ae7fad-c358-48f8-ac24-e3d236a6b7c5\") "
Sep 10 21:24:12.620686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620501    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31ae7fad-c358-48f8-ac24-e3d236a6b7c5" (UID: "31ae7fad-c358-48f8-ac24-e3d236a6b7c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:24:12.620686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620516    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31ae7fad-c358-48f8-ac24-e3d236a6b7c5" (UID: "31ae7fad-c358-48f8-ac24-e3d236a6b7c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:24:12.620686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620654    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31ae7fad-c358-48f8-ac24-e3d236a6b7c5" (UID: "31ae7fad-c358-48f8-ac24-e3d236a6b7c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:24:12.621003 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620841    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:12.621003 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620851    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31ae7fad-c358-48f8-ac24-e3d236a6b7c5" (UID: "31ae7fad-c358-48f8-ac24-e3d236a6b7c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:12.621003 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620860    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:12.621003 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620911    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:12.621003 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.620929    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:12.622569 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.622544    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31ae7fad-c358-48f8-ac24-e3d236a6b7c5" (UID: "31ae7fad-c358-48f8-ac24-e3d236a6b7c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:24:12.622667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.622644    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-kube-api-access-l6x82" (OuterVolumeSpecName: "kube-api-access-l6x82") pod "31ae7fad-c358-48f8-ac24-e3d236a6b7c5" (UID: "31ae7fad-c358-48f8-ac24-e3d236a6b7c5"). InnerVolumeSpecName "kube-api-access-l6x82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:12.622825 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.622804    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "31ae7fad-c358-48f8-ac24-e3d236a6b7c5" (UID: "31ae7fad-c358-48f8-ac24-e3d236a6b7c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:12.622872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.622804    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31ae7fad-c358-48f8-ac24-e3d236a6b7c5" (UID: "31ae7fad-c358-48f8-ac24-e3d236a6b7c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:24:12.721991 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.721957    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:12.721991 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.721986    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:12.721991 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.721995    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6x82\" (UniqueName: \"kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-kube-api-access-l6x82\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:12.722228 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.722005    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:12.722228 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.722014    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ae7fad-c358-48f8-ac24-e3d236a6b7c5-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:12.819910 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.819874    2573 generic.go:356] "Generic (PLEG): container finished" podID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerID="fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54" exitCode=0
Sep 10 21:24:12.820119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.819924    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"
Sep 10 21:24:12.820119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.819967    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" event={"ID":"31ae7fad-c358-48f8-ac24-e3d236a6b7c5","Type":"ContainerDied","Data":"fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54"}
Sep 10 21:24:12.820119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.820022    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk" event={"ID":"31ae7fad-c358-48f8-ac24-e3d236a6b7c5","Type":"ContainerDied","Data":"17ac2b2f49b99181144e7ca459419f1d829a8966844744439f779acc87a1779a"}
Sep 10 21:24:12.820119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.820045    2573 scope.go:117] "RemoveContainer" containerID="fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54"
Sep 10 21:24:12.829538 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.829521    2573 scope.go:117] "RemoveContainer" containerID="90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d"
Sep 10 21:24:12.837937 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.837916    2573 scope.go:117] "RemoveContainer" containerID="fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54"
Sep 10 21:24:12.838225 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:24:12.838196    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54\": container with ID starting with fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54 not found: ID does not exist" containerID="fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54"
Sep 10 21:24:12.838274 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.838234    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54"} err="failed to get container status \"fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54\": rpc error: code = NotFound desc = could not find container \"fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54\": container with ID starting with fb888a5e9ecb28cff44b359fde7873983338534f2a4c3654eb1d2c0a84257c54 not found: ID does not exist"
Sep 10 21:24:12.838274 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.838255    2573 scope.go:117] "RemoveContainer" containerID="90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d"
Sep 10 21:24:12.838517 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:24:12.838498    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d\": container with ID starting with 90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d not found: ID does not exist" containerID="90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d"
Sep 10 21:24:12.838571 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.838527    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d"} err="failed to get container status \"90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d\": rpc error: code = NotFound desc = could not find container \"90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d\": container with ID starting with 90a6a2158c0ce58f51b6b359af7c4d90fe9b39ca1759eb5720617e0bee85e58d not found: ID does not exist"
Sep 10 21:24:12.846101 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.846078    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"]
Sep 10 21:24:12.855456 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:12.855432    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-780a0cdd-openshift-default-558c889d46-vkxtk"]
Sep 10 21:24:14.685535 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:14.685500    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" path="/var/lib/kubelet/pods/31ae7fad-c358-48f8-ac24-e3d236a6b7c5/volumes"
Sep 10 21:24:19.799096 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.799049    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"]
Sep 10 21:24:19.799610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.799438    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy"
Sep 10 21:24:19.799610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.799450    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy"
Sep 10 21:24:19.799610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.799522    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy"
Sep 10 21:24:19.801999 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.801981    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.803981 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.803955    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-f135f4\"/\"default-dockercfg-lfhzk\""
Sep 10 21:24:19.804838 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.804818    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-f135f4\"/\"openshift-service-ca.crt\""
Sep 10 21:24:19.804959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.804838    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-f135f4\"/\"autores-model-f135f4-kserve-self-signed-certs\""
Sep 10 21:24:19.804959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.804844    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-f135f4\"/\"kube-root-ca.crt\""
Sep 10 21:24:19.810865 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.810839    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"]
Sep 10 21:24:19.894227 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.894183    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwpck\" (UniqueName: \"kubernetes.io/projected/66ea47f7-cef2-4a51-b376-f792bb652b46-kube-api-access-fwpck\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.894576 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.894549    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66ea47f7-cef2-4a51-b376-f792bb652b46-tls-certs\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.894749 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.894731    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-tmp-dir\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.894928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.894910    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-dshm\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.895065 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.895050    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-model-cache\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.895184 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.895169    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-home\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.995937 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.995893    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66ea47f7-cef2-4a51-b376-f792bb652b46-tls-certs\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.996150 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.995958    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-tmp-dir\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.996150 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.996000    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-dshm\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.996150 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.996036    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-model-cache\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.996150 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.996068    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-home\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.996150 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.996133    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwpck\" (UniqueName: \"kubernetes.io/projected/66ea47f7-cef2-4a51-b376-f792bb652b46-kube-api-access-fwpck\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.996504 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.996475    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-tmp-dir\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.996593 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.996484    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-home\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.996652 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.996625    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-model-cache\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.998764 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.998366    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-dshm\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:19.998764 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:19.998732    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66ea47f7-cef2-4a51-b376-f792bb652b46-tls-certs\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:20.004178 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:20.004155    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwpck\" (UniqueName: \"kubernetes.io/projected/66ea47f7-cef2-4a51-b376-f792bb652b46-kube-api-access-fwpck\") pod \"autores-model-f135f4-kserve-796d79cf67-27k4s\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") " pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:20.112384 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:20.112346    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:20.393173 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:20.390508    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"]
Sep 10 21:24:20.867980 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:20.867939    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s" event={"ID":"66ea47f7-cef2-4a51-b376-f792bb652b46","Type":"ContainerStarted","Data":"b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333"}
Sep 10 21:24:20.868347 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:20.867995    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s" event={"ID":"66ea47f7-cef2-4a51-b376-f792bb652b46","Type":"ContainerStarted","Data":"ee03ea44185e000840a0b7cb2a7c52f3e58ef621dc78af7fb42164b1fa526aed"}
Sep 10 21:24:20.895575 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:20.895504    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s" podStartSLOduration=1.895483263 podStartE2EDuration="1.895483263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:20.891571061 +0000 UTC m=+836.825237877" watchObservedRunningTime="2026-09-10 21:24:20.895483263 +0000 UTC m=+836.829150078"
Sep 10 21:24:23.416224 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:23.416184    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:24:24.893314 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:24.893282    2573 generic.go:356] "Generic (PLEG): container finished" podID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerID="d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5" exitCode=0
Sep 10 21:24:24.893930 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:24.893362    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" event={"ID":"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa","Type":"ContainerDied","Data":"d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5"}
Sep 10 21:24:24.893930 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:24.893729    2573 scope.go:117] "RemoveContainer" containerID="d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5"
Sep 10 21:24:24.894867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:24.894836    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:24.894988 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:24.894905    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:24.894988 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:24.894940    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:25.808668 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:25.808610    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:25.903289 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:25.903254    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" event={"ID":"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa","Type":"ContainerStarted","Data":"f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a"}
Sep 10 21:24:26.497211 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:26.497177    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:26.498256 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:26.498224    2573 patch_prober.go:28] interesting pod/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 10 21:24:26.498372 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:26.498292    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 10 21:24:27.497840 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:27.497800    2573 patch_prober.go:28] interesting pod/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 10 21:24:27.498237 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:27.497872    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 10 21:24:28.497709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:28.497663    2573 patch_prober.go:28] interesting pod/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 10 21:24:28.497944 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:28.497719    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 10 21:24:29.497722 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:29.497678    2573 patch_prober.go:28] interesting pod/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 10 21:24:29.497930 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:29.497766    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 10 21:24:30.113442 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:30.113392    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:30.113858 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:30.113460    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:30.125744 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:30.125716    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:30.497611 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:30.497520    2573 patch_prober.go:28] interesting pod/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 10 21:24:30.497611 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:30.497588    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 10 21:24:30.942392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:30.942362    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:24:31.501212 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:31.501182    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:31.501592 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:31.501440    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:31.502099 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:31.502081    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:36.211396 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.211359    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"]
Sep 10 21:24:36.211819 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.211752    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy"
Sep 10 21:24:36.211819 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.211765    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy"
Sep 10 21:24:36.211903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.211869    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ae7fad-c358-48f8-ac24-e3d236a6b7c5" containerName="istio-proxy"
Sep 10 21:24:36.216699 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.216677    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"
Sep 10 21:24:36.218757 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.218738    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:24:36.223188 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.223159    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"]
Sep 10 21:24:36.273992 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.273955    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7114bdb-6a69-46d0-9d76-91420f9ffcb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg\" (UID: \"a7114bdb-6a69-46d0-9d76-91420f9ffcb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"
Sep 10 21:24:36.375480 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.375441    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7114bdb-6a69-46d0-9d76-91420f9ffcb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg\" (UID: \"a7114bdb-6a69-46d0-9d76-91420f9ffcb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"
Sep 10 21:24:36.376178 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.376152    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7114bdb-6a69-46d0-9d76-91420f9ffcb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg\" (UID: \"a7114bdb-6a69-46d0-9d76-91420f9ffcb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"
Sep 10 21:24:36.528895 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.528804    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"
Sep 10 21:24:36.799657 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.799632    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"]
Sep 10 21:24:36.801532 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:24:36.801499    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7114bdb_6a69_46d0_9d76_91420f9ffcb1.slice/crio-e88a33873f50cf6df5bd449d4fd1c19ebe26233321ce7bffd620611157f2e9ef WatchSource:0}: Error finding container e88a33873f50cf6df5bd449d4fd1c19ebe26233321ce7bffd620611157f2e9ef: Status 404 returned error can't find the container with id e88a33873f50cf6df5bd449d4fd1c19ebe26233321ce7bffd620611157f2e9ef
Sep 10 21:24:36.970683 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:36.970647    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg" event={"ID":"a7114bdb-6a69-46d0-9d76-91420f9ffcb1","Type":"ContainerStarted","Data":"e88a33873f50cf6df5bd449d4fd1c19ebe26233321ce7bffd620611157f2e9ef"}
Sep 10 21:24:37.977169 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:37.977132    2573 generic.go:356] "Generic (PLEG): container finished" podID="a7114bdb-6a69-46d0-9d76-91420f9ffcb1" containerID="63e06bd4e5564f3944b59297fac3863556756e2b7c5dde402d4617288efa5cb1" exitCode=0
Sep 10 21:24:37.977561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:37.977190    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg" event={"ID":"a7114bdb-6a69-46d0-9d76-91420f9ffcb1","Type":"ContainerDied","Data":"63e06bd4e5564f3944b59297fac3863556756e2b7c5dde402d4617288efa5cb1"}
Sep 10 21:24:39.114991 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:39.114966    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"
Sep 10 21:24:39.201658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:39.201618    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7114bdb-6a69-46d0-9d76-91420f9ffcb1-maas-api-service-ca\") pod \"a7114bdb-6a69-46d0-9d76-91420f9ffcb1\" (UID: \"a7114bdb-6a69-46d0-9d76-91420f9ffcb1\") "
Sep 10 21:24:39.202008 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:39.201985    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7114bdb-6a69-46d0-9d76-91420f9ffcb1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a7114bdb-6a69-46d0-9d76-91420f9ffcb1" (UID: "a7114bdb-6a69-46d0-9d76-91420f9ffcb1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:39.302485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:39.302399    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7114bdb-6a69-46d0-9d76-91420f9ffcb1-maas-api-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:39.989405 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:39.989378    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"
Sep 10 21:24:39.989576 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:39.989420    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg" event={"ID":"a7114bdb-6a69-46d0-9d76-91420f9ffcb1","Type":"ContainerDied","Data":"e88a33873f50cf6df5bd449d4fd1c19ebe26233321ce7bffd620611157f2e9ef"}
Sep 10 21:24:39.989576 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:39.989470    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e88a33873f50cf6df5bd449d4fd1c19ebe26233321ce7bffd620611157f2e9ef"
Sep 10 21:24:40.823016 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:40.822972    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:24:42.564130 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:42.564093    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"]
Sep 10 21:24:42.572207 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:42.572167    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23d91506-ch9bg"]
Sep 10 21:24:42.680606 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:42.680575    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7114bdb-6a69-46d0-9d76-91420f9ffcb1" path="/var/lib/kubelet/pods/a7114bdb-6a69-46d0-9d76-91420f9ffcb1/volumes"
Sep 10 21:24:42.692476 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:42.692440    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"]
Sep 10 21:24:42.692729 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:42.692698    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy" containerID="cri-o://f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a" gracePeriod=30
Sep 10 21:24:43.403030 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.402989    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"]
Sep 10 21:24:43.403538 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.403520    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7114bdb-6a69-46d0-9d76-91420f9ffcb1" containerName="revoke-keys"
Sep 10 21:24:43.403585 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.403543    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7114bdb-6a69-46d0-9d76-91420f9ffcb1" containerName="revoke-keys"
Sep 10 21:24:43.403651 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.403641    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7114bdb-6a69-46d0-9d76-91420f9ffcb1" containerName="revoke-keys"
Sep 10 21:24:43.413962 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.413935    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.416351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.416326    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-dc873af1-openshift-default-dockercfg-7cxnl\""
Sep 10 21:24:43.417965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.417943    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"]
Sep 10 21:24:43.545938 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.545893    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-token\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.546143 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.545949    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-socket\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.546143 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.545965    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istiod-ca-cert\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.546143 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.546057    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jh2c\" (UniqueName: \"kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-kube-api-access-7jh2c\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.546143 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.546114    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-data\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.546329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.546165    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-certs\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.546329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.546208    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-envoy\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.546329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.546228    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-podinfo\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.546329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.546295    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-credential-socket\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.647087 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647051    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-socket\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.647087 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647086    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istiod-ca-cert\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.647613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647114    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jh2c\" (UniqueName: \"kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-kube-api-access-7jh2c\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.647613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647238    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-data\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.647613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647282    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-certs\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.647613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647406    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-envoy\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.647613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647431    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-podinfo\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.647613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647500    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-socket\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.647613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647517    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-credential-socket\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.647613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647599    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-token\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.648072 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647624    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-certs\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.648072 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647655    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-data\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.648072 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.647977    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istiod-ca-cert\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.648072 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.648045    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-credential-socket\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.649830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.649807    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-envoy\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.650056 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.650033    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-podinfo\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.655374 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.655311    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-token\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.655493 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.655452    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jh2c\" (UniqueName: \"kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-kube-api-access-7jh2c\") pod \"e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") " pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:43.725392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:43.725354    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:44.000997 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:44.000965    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"]
Sep 10 21:24:44.002365 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:24:44.002331    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0632efc6_ccc8_4b43_bb9c_26324dbe400d.slice/crio-f8bc8192f714b00a5363924f926b600147bc4b86a1fe23665a0006939e2a7fae WatchSource:0}: Error finding container f8bc8192f714b00a5363924f926b600147bc4b86a1fe23665a0006939e2a7fae: Status 404 returned error can't find the container with id f8bc8192f714b00a5363924f926b600147bc4b86a1fe23665a0006939e2a7fae
Sep 10 21:24:44.004881 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:44.004848    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:44.004959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:44.004916    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:44.004959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:44.004943    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:24:44.012114 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:44.012084    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" event={"ID":"0632efc6-ccc8-4b43-bb9c-26324dbe400d","Type":"ContainerStarted","Data":"f8bc8192f714b00a5363924f926b600147bc4b86a1fe23665a0006939e2a7fae"}
Sep 10 21:24:45.019307 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:45.019266    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" event={"ID":"0632efc6-ccc8-4b43-bb9c-26324dbe400d","Type":"ContainerStarted","Data":"4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607"}
Sep 10 21:24:45.041569 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:45.041495    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" podStartSLOduration=2.041474784 podStartE2EDuration="2.041474784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:45.040660178 +0000 UTC m=+860.974326993" watchObservedRunningTime="2026-09-10 21:24:45.041474784 +0000 UTC m=+860.975141600"
Sep 10 21:24:45.726100 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:45.726060    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:45.731000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:45.730975    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:46.025001 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:46.024908    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:46.025965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:46.025944    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:24:46.503803 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:46.503747    2573 patch_prober.go:28] interesting pod/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:24:46.503971 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:46.503826    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:24:47.945228 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:47.945194    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:48.037322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.037244    2573 generic.go:356] "Generic (PLEG): container finished" podID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerID="f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a" exitCode=0
Sep 10 21:24:48.037322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.037293    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"
Sep 10 21:24:48.037503 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.037327    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" event={"ID":"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa","Type":"ContainerDied","Data":"f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a"}
Sep 10 21:24:48.037503 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.037382    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z" event={"ID":"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa","Type":"ContainerDied","Data":"ca723a0d99a6899b297b9a577fb1515ba0b872f074499fbe47c6f24f9a35b1d9"}
Sep 10 21:24:48.037503 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.037412    2573 scope.go:117] "RemoveContainer" containerID="f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a"
Sep 10 21:24:48.046524 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.046507    2573 scope.go:117] "RemoveContainer" containerID="d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5"
Sep 10 21:24:48.055071 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.055052    2573 scope.go:117] "RemoveContainer" containerID="f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a"
Sep 10 21:24:48.055353 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:24:48.055333    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a\": container with ID starting with f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a not found: ID does not exist" containerID="f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a"
Sep 10 21:24:48.055431 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.055368    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a"} err="failed to get container status \"f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a\": rpc error: code = NotFound desc = could not find container \"f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a\": container with ID starting with f32160cbab12068c57cba1cce8c82d133e3aeff424b7ab6ee624f048bd86b44a not found: ID does not exist"
Sep 10 21:24:48.055431 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.055394    2573 scope.go:117] "RemoveContainer" containerID="d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5"
Sep 10 21:24:48.055653 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:24:48.055637    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5\": container with ID starting with d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5 not found: ID does not exist" containerID="d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5"
Sep 10 21:24:48.055696 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.055660    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5"} err="failed to get container status \"d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5\": rpc error: code = NotFound desc = could not find container \"d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5\": container with ID starting with d0927a72e2b6d97b73b65a090c568804d48f2801ea38697d2e8f1e1b7ae12ed5 not found: ID does not exist"
Sep 10 21:24:48.094032 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.093996    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-envoy\") pod \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") "
Sep 10 21:24:48.094193 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094051    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-socket\") pod \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") "
Sep 10 21:24:48.094193 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094082    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-credential-socket\") pod \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") "
Sep 10 21:24:48.094193 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094102    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-certs\") pod \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") "
Sep 10 21:24:48.094193 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094119    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-podinfo\") pod \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") "
Sep 10 21:24:48.094193 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094154    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istiod-ca-cert\") pod \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") "
Sep 10 21:24:48.094193 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094191    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-token\") pod \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") "
Sep 10 21:24:48.094496 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094241    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-data\") pod \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") "
Sep 10 21:24:48.094496 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094288    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-422nq\" (UniqueName: \"kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-kube-api-access-422nq\") pod \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\" (UID: \"e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa\") "
Sep 10 21:24:48.094496 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094327    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" (UID: "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:24:48.094496 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094437    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" (UID: "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:24:48.094705 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094588    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-data" (OuterVolumeSpecName: "istio-data") pod "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" (UID: "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:24:48.094705 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094615    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" (UID: "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:48.094903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094869    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:48.094903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094893    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:48.094903 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094902    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:48.095089 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094910    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:48.095089 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.094956    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" (UID: "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:24:48.096462 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.096429    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-token" (OuterVolumeSpecName: "istio-token") pod "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" (UID: "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:48.096573 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.096489    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" (UID: "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:24:48.096573 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.096562    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-kube-api-access-422nq" (OuterVolumeSpecName: "kube-api-access-422nq") pod "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" (UID: "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa"). InnerVolumeSpecName "kube-api-access-422nq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:48.096708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.096590    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" (UID: "e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:24:48.195543 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.195506    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:48.195543 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.195545    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:48.195747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.195562    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:48.195747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.195576    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:48.195747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.195591    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-422nq\" (UniqueName: \"kubernetes.io/projected/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa-kube-api-access-422nq\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:48.372399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.372363    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"]
Sep 10 21:24:48.384608 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.384575    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-23d91506-gw-openshift-default-7477564bcb-l9q5z"]
Sep 10 21:24:48.685195 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:48.685099    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" path="/var/lib/kubelet/pods/e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa/volumes"
Sep 10 21:24:52.713175 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.713138    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"]
Sep 10 21:24:52.713544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.713529    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy"
Sep 10 21:24:52.713544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.713545    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy"
Sep 10 21:24:52.713650 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.713557    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy"
Sep 10 21:24:52.713650 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.713563    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy"
Sep 10 21:24:52.713650 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.713634    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy"
Sep 10 21:24:52.713650 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.713642    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8a65cd5-aa12-4280-87a3-9a8e96bf6ffa" containerName="istio-proxy"
Sep 10 21:24:52.715922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.715901    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"
Sep 10 21:24:52.718441 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.718416    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:24:52.726376 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.726349    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"]
Sep 10 21:24:52.840324 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.840288    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d428c7-8614-40de-a547-d83663a0dfe2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-f135f4-847j5\" (UID: \"27d428c7-8614-40de-a547-d83663a0dfe2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"
Sep 10 21:24:52.941379 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.941341    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d428c7-8614-40de-a547-d83663a0dfe2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-f135f4-847j5\" (UID: \"27d428c7-8614-40de-a547-d83663a0dfe2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"
Sep 10 21:24:52.942022 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:52.942002    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d428c7-8614-40de-a547-d83663a0dfe2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-f135f4-847j5\" (UID: \"27d428c7-8614-40de-a547-d83663a0dfe2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"
Sep 10 21:24:53.029649 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:53.029553    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"
Sep 10 21:24:53.306881 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:53.306854    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"]
Sep 10 21:24:53.308770 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:24:53.308742    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d428c7_8614_40de_a547_d83663a0dfe2.slice/crio-28c9a4830dee3385167ea3eaaaff482eae485decfa925b35e328c9a5883e6cb8 WatchSource:0}: Error finding container 28c9a4830dee3385167ea3eaaaff482eae485decfa925b35e328c9a5883e6cb8: Status 404 returned error can't find the container with id 28c9a4830dee3385167ea3eaaaff482eae485decfa925b35e328c9a5883e6cb8
Sep 10 21:24:54.076590 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:54.076553    2573 generic.go:356] "Generic (PLEG): container finished" podID="27d428c7-8614-40de-a547-d83663a0dfe2" containerID="7cedae263f6e98617c88f7d4f112facedcbc35d9ed416ef017a8ac702e51c085" exitCode=0
Sep 10 21:24:54.077000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:54.076635    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5" event={"ID":"27d428c7-8614-40de-a547-d83663a0dfe2","Type":"ContainerDied","Data":"7cedae263f6e98617c88f7d4f112facedcbc35d9ed416ef017a8ac702e51c085"}
Sep 10 21:24:54.077000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:54.076688    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5" event={"ID":"27d428c7-8614-40de-a547-d83663a0dfe2","Type":"ContainerStarted","Data":"28c9a4830dee3385167ea3eaaaff482eae485decfa925b35e328c9a5883e6cb8"}
Sep 10 21:24:55.214098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:55.214075    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"
Sep 10 21:24:55.362306 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:55.362205    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d428c7-8614-40de-a547-d83663a0dfe2-maas-api-service-ca\") pod \"27d428c7-8614-40de-a547-d83663a0dfe2\" (UID: \"27d428c7-8614-40de-a547-d83663a0dfe2\") "
Sep 10 21:24:55.362574 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:55.362551    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27d428c7-8614-40de-a547-d83663a0dfe2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27d428c7-8614-40de-a547-d83663a0dfe2" (UID: "27d428c7-8614-40de-a547-d83663a0dfe2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:55.463406 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:55.463374    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d428c7-8614-40de-a547-d83663a0dfe2-maas-api-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:24:56.096198 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:56.096166    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"
Sep 10 21:24:56.096392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:56.096207    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5" event={"ID":"27d428c7-8614-40de-a547-d83663a0dfe2","Type":"ContainerDied","Data":"28c9a4830dee3385167ea3eaaaff482eae485decfa925b35e328c9a5883e6cb8"}
Sep 10 21:24:56.096392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:56.096287    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28c9a4830dee3385167ea3eaaaff482eae485decfa925b35e328c9a5883e6cb8"
Sep 10 21:24:57.620575 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:24:57.620538    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:25:03.620509 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:03.620473    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"]
Sep 10 21:25:03.622887 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:03.622856    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f135f4-847j5"]
Sep 10 21:25:04.689124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:04.689093    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27d428c7-8614-40de-a547-d83663a0dfe2" path="/var/lib/kubelet/pods/27d428c7-8614-40de-a547-d83663a0dfe2/volumes"
Sep 10 21:25:08.168278 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:08.168194    2573 generic.go:356] "Generic (PLEG): container finished" podID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerID="4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607" exitCode=0
Sep 10 21:25:08.168278 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:08.168266    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" event={"ID":"0632efc6-ccc8-4b43-bb9c-26324dbe400d","Type":"ContainerDied","Data":"4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607"}
Sep 10 21:25:08.168724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:08.168671    2573 scope.go:117] "RemoveContainer" containerID="4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607"
Sep 10 21:25:08.169716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:08.169692    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:08.169803 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:08.169755    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:08.169850 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:08.169803    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:08.956204 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:08.956168    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"]
Sep 10 21:25:09.178709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:09.178665    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" event={"ID":"0632efc6-ccc8-4b43-bb9c-26324dbe400d","Type":"ContainerStarted","Data":"b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e"}
Sep 10 21:25:09.726074 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:09.726041    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:25:09.726248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:09.726225    2573 patch_prober.go:28] interesting pod/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:25:09.726294 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:09.726270    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:25:10.726247 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:10.726210    2573 patch_prober.go:28] interesting pod/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:25:10.726630 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:10.726272    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:25:11.726046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:11.726010    2573 patch_prober.go:28] interesting pod/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:25:11.726217 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:11.726061    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:25:12.726732 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:12.726701    2573 patch_prober.go:28] interesting pod/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:25:12.727134 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:12.726749    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:25:13.725741 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:13.725701    2573 patch_prober.go:28] interesting pod/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:25:13.725944 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:13.725762    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:25:14.609112 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:14.609069    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:25:14.902970 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:14.902865    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:25:14.903283 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:14.903258    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:25:14.904409 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:14.904385    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:25:23.975848 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:23.975746    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s" podUID="66ea47f7-cef2-4a51-b376-f792bb652b46" containerName="main" containerID="cri-o://b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333" gracePeriod=45
Sep 10 21:25:24.225047 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.225019    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:25:24.281301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.281212    2573 generic.go:356] "Generic (PLEG): container finished" podID="66ea47f7-cef2-4a51-b376-f792bb652b46" containerID="b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333" exitCode=0
Sep 10 21:25:24.281301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.281254    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"
Sep 10 21:25:24.281301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.281274    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s" event={"ID":"66ea47f7-cef2-4a51-b376-f792bb652b46","Type":"ContainerDied","Data":"b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333"}
Sep 10 21:25:24.281545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.281311    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s" event={"ID":"66ea47f7-cef2-4a51-b376-f792bb652b46","Type":"ContainerDied","Data":"ee03ea44185e000840a0b7cb2a7c52f3e58ef621dc78af7fb42164b1fa526aed"}
Sep 10 21:25:24.281545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.281333    2573 scope.go:117] "RemoveContainer" containerID="b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333"
Sep 10 21:25:24.290628 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.290606    2573 scope.go:117] "RemoveContainer" containerID="b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333"
Sep 10 21:25:24.290945 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:25:24.290925    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333\": container with ID starting with b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333 not found: ID does not exist" containerID="b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333"
Sep 10 21:25:24.291030 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.290951    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333"} err="failed to get container status \"b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333\": rpc error: code = NotFound desc = could not find container \"b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333\": container with ID starting with b3e99e05a43e0a4f92434167485b9b4a4605ade8d8cc0d65f419c0b81dd62333 not found: ID does not exist"
Sep 10 21:25:24.325799 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.325744    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-dshm\") pod \"66ea47f7-cef2-4a51-b376-f792bb652b46\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") "
Sep 10 21:25:24.325799 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.325806    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-tmp-dir\") pod \"66ea47f7-cef2-4a51-b376-f792bb652b46\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") "
Sep 10 21:25:24.326052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.325848    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwpck\" (UniqueName: \"kubernetes.io/projected/66ea47f7-cef2-4a51-b376-f792bb652b46-kube-api-access-fwpck\") pod \"66ea47f7-cef2-4a51-b376-f792bb652b46\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") "
Sep 10 21:25:24.326052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.325880    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-model-cache\") pod \"66ea47f7-cef2-4a51-b376-f792bb652b46\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") "
Sep 10 21:25:24.326052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.325959    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-home\") pod \"66ea47f7-cef2-4a51-b376-f792bb652b46\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") "
Sep 10 21:25:24.326052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.325986    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66ea47f7-cef2-4a51-b376-f792bb652b46-tls-certs\") pod \"66ea47f7-cef2-4a51-b376-f792bb652b46\" (UID: \"66ea47f7-cef2-4a51-b376-f792bb652b46\") "
Sep 10 21:25:24.326257 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.326131    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "66ea47f7-cef2-4a51-b376-f792bb652b46" (UID: "66ea47f7-cef2-4a51-b376-f792bb652b46"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:24.326257 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.326177    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-model-cache" (OuterVolumeSpecName: "model-cache") pod "66ea47f7-cef2-4a51-b376-f792bb652b46" (UID: "66ea47f7-cef2-4a51-b376-f792bb652b46"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:24.326257 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.326239    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-home" (OuterVolumeSpecName: "home") pod "66ea47f7-cef2-4a51-b376-f792bb652b46" (UID: "66ea47f7-cef2-4a51-b376-f792bb652b46"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:24.326379 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.326367    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:24.326417 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.326387    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:24.326417 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.326401    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:24.328125 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.328080    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66ea47f7-cef2-4a51-b376-f792bb652b46-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "66ea47f7-cef2-4a51-b376-f792bb652b46" (UID: "66ea47f7-cef2-4a51-b376-f792bb652b46"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:25:24.328221 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.328136    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-dshm" (OuterVolumeSpecName: "dshm") pod "66ea47f7-cef2-4a51-b376-f792bb652b46" (UID: "66ea47f7-cef2-4a51-b376-f792bb652b46"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:24.328221 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.328143    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66ea47f7-cef2-4a51-b376-f792bb652b46-kube-api-access-fwpck" (OuterVolumeSpecName: "kube-api-access-fwpck") pod "66ea47f7-cef2-4a51-b376-f792bb652b46" (UID: "66ea47f7-cef2-4a51-b376-f792bb652b46"). InnerVolumeSpecName "kube-api-access-fwpck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:24.426820 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.426760    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66ea47f7-cef2-4a51-b376-f792bb652b46-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:24.426820 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.426813    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwpck\" (UniqueName: \"kubernetes.io/projected/66ea47f7-cef2-4a51-b376-f792bb652b46-kube-api-access-fwpck\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:24.426820 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.426824    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66ea47f7-cef2-4a51-b376-f792bb652b46-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:24.512430 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.512404    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:25:24.513572 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.513551    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:25:24.606015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.605979    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"]
Sep 10 21:25:24.610161 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.610137    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-f135f4/autores-model-f135f4-kserve-796d79cf67-27k4s"]
Sep 10 21:25:24.680853 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:24.680822    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66ea47f7-cef2-4a51-b376-f792bb652b46" path="/var/lib/kubelet/pods/66ea47f7-cef2-4a51-b376-f792bb652b46/volumes"
Sep 10 21:25:26.111529 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:26.111494    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:25:29.722738 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:29.722702    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"]
Sep 10 21:25:29.723258 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:29.722952    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy" containerID="cri-o://b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e" gracePeriod=30
Sep 10 21:25:29.905667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:29.905628    2573 patch_prober.go:28] interesting pod/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:25:29.905856 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:29.905697    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:25:30.425972 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.425928    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"]
Sep 10 21:25:30.426307 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.426293    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66ea47f7-cef2-4a51-b376-f792bb652b46" containerName="main"
Sep 10 21:25:30.426307 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.426307    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="66ea47f7-cef2-4a51-b376-f792bb652b46" containerName="main"
Sep 10 21:25:30.426424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.426322    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27d428c7-8614-40de-a547-d83663a0dfe2" containerName="revoke-keys"
Sep 10 21:25:30.426424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.426328    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="27d428c7-8614-40de-a547-d83663a0dfe2" containerName="revoke-keys"
Sep 10 21:25:30.426424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.426386    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="66ea47f7-cef2-4a51-b376-f792bb652b46" containerName="main"
Sep 10 21:25:30.426424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.426395    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="27d428c7-8614-40de-a547-d83663a0dfe2" containerName="revoke-keys"
Sep 10 21:25:30.436412 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.436375    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.438452 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.438429    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-fd195260-openshift-default-dockercfg-fthpn\""
Sep 10 21:25:30.442721 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.442694    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"]
Sep 10 21:25:30.485275 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.485239    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-data\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.485275 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.485280    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-token\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.485553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.485307    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-envoy\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.485553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.485357    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-certs\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.485553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.485448    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-podinfo\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.485553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.485477    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf3ef09f-8588-4a82-902a-6407fbb8939f-istiod-ca-cert\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.485553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.485525    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-credential-socket\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.485765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.485578    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-socket\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.485765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.485618    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cvmq\" (UniqueName: \"kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-kube-api-access-2cvmq\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587074 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587036    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-podinfo\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587074 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587076    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf3ef09f-8588-4a82-902a-6407fbb8939f-istiod-ca-cert\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587112    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-credential-socket\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587140    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-socket\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587164    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cvmq\" (UniqueName: \"kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-kube-api-access-2cvmq\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587184    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-data\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587207    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-token\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587258    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-envoy\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587308    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-certs\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587753 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587711    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-credential-socket\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587753 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587723    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-certs\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587849    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-socket\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.587917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.587877    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf3ef09f-8588-4a82-902a-6407fbb8939f-istiod-ca-cert\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.588026 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.588012    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-data\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.589663 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.589636    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-envoy\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.589790 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.589748    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-podinfo\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.594451 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.594429    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-token\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.594552 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.594536    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cvmq\" (UniqueName: \"kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-kube-api-access-2cvmq\") pod \"e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") " pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:30.748981 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:30.748899    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:31.021079 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:31.021054    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"]
Sep 10 21:25:31.023003 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:25:31.022978    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3ef09f_8588_4a82_902a_6407fbb8939f.slice/crio-9df1508bebf03f87156d10890a8f4d877eed22b5a18d6d0302e136b32c405b3e WatchSource:0}: Error finding container 9df1508bebf03f87156d10890a8f4d877eed22b5a18d6d0302e136b32c405b3e: Status 404 returned error can't find the container with id 9df1508bebf03f87156d10890a8f4d877eed22b5a18d6d0302e136b32c405b3e
Sep 10 21:25:31.025806 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:31.025760    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:31.025864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:31.025841    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:31.025902 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:31.025871    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:31.318958 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:31.318875    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" event={"ID":"cf3ef09f-8588-4a82-902a-6407fbb8939f","Type":"ContainerStarted","Data":"0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b"}
Sep 10 21:25:31.318958 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:31.318926    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" event={"ID":"cf3ef09f-8588-4a82-902a-6407fbb8939f","Type":"ContainerStarted","Data":"9df1508bebf03f87156d10890a8f4d877eed22b5a18d6d0302e136b32c405b3e"}
Sep 10 21:25:31.340909 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:31.340856    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" podStartSLOduration=1.340838307 podStartE2EDuration="1.340838307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:31.339643048 +0000 UTC m=+907.273309863" watchObservedRunningTime="2026-09-10 21:25:31.340838307 +0000 UTC m=+907.274505123"
Sep 10 21:25:31.749084 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:31.749050    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:32.754500 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:32.754469    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:33.330056 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:33.330025    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:33.331120 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:33.331094    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:34.984048 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:34.984020    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:25:35.030815 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.030692    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-socket\") pod \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") "
Sep 10 21:25:35.030815 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.030731    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-certs\") pod \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") "
Sep 10 21:25:35.030815 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.030806    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-podinfo\") pod \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") "
Sep 10 21:25:35.031113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.030885    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-credential-socket\") pod \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") "
Sep 10 21:25:35.031113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.030932    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-envoy\") pod \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") "
Sep 10 21:25:35.031113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.030969    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jh2c\" (UniqueName: \"kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-kube-api-access-7jh2c\") pod \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") "
Sep 10 21:25:35.031113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.030998    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-data\") pod \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") "
Sep 10 21:25:35.031113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031047    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0632efc6-ccc8-4b43-bb9c-26324dbe400d" (UID: "0632efc6-ccc8-4b43-bb9c-26324dbe400d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:35.031113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031072    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istiod-ca-cert\") pod \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") "
Sep 10 21:25:35.031113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031075    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0632efc6-ccc8-4b43-bb9c-26324dbe400d" (UID: "0632efc6-ccc8-4b43-bb9c-26324dbe400d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:35.031113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031105    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-token\") pod \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\" (UID: \"0632efc6-ccc8-4b43-bb9c-26324dbe400d\") "
Sep 10 21:25:35.031608 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031182    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0632efc6-ccc8-4b43-bb9c-26324dbe400d" (UID: "0632efc6-ccc8-4b43-bb9c-26324dbe400d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:35.031608 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031316    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-data" (OuterVolumeSpecName: "istio-data") pod "0632efc6-ccc8-4b43-bb9c-26324dbe400d" (UID: "0632efc6-ccc8-4b43-bb9c-26324dbe400d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:35.031608 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031459    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:35.031608 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031480    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:35.031608 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031494    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:35.031608 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031506    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:35.031608 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.031576    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0632efc6-ccc8-4b43-bb9c-26324dbe400d" (UID: "0632efc6-ccc8-4b43-bb9c-26324dbe400d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:35.033107 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.033084    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0632efc6-ccc8-4b43-bb9c-26324dbe400d" (UID: "0632efc6-ccc8-4b43-bb9c-26324dbe400d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:25:35.033322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.033297    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0632efc6-ccc8-4b43-bb9c-26324dbe400d" (UID: "0632efc6-ccc8-4b43-bb9c-26324dbe400d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:35.033651 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.033627    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-kube-api-access-7jh2c" (OuterVolumeSpecName: "kube-api-access-7jh2c") pod "0632efc6-ccc8-4b43-bb9c-26324dbe400d" (UID: "0632efc6-ccc8-4b43-bb9c-26324dbe400d"). InnerVolumeSpecName "kube-api-access-7jh2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:35.033764 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.033654    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-token" (OuterVolumeSpecName: "istio-token") pod "0632efc6-ccc8-4b43-bb9c-26324dbe400d" (UID: "0632efc6-ccc8-4b43-bb9c-26324dbe400d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:35.132823 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.132766    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:35.132823 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.132815    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:35.132823 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.132824    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jh2c\" (UniqueName: \"kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-kube-api-access-7jh2c\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:35.132823 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.132834    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:35.133092 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.132842    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0632efc6-ccc8-4b43-bb9c-26324dbe400d-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:35.343364 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.343327    2573 generic.go:356] "Generic (PLEG): container finished" podID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerID="b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e" exitCode=0
Sep 10 21:25:35.344809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.343930    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"
Sep 10 21:25:35.345144 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.345111    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" event={"ID":"0632efc6-ccc8-4b43-bb9c-26324dbe400d","Type":"ContainerDied","Data":"b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e"}
Sep 10 21:25:35.345307 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.345222    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8" event={"ID":"0632efc6-ccc8-4b43-bb9c-26324dbe400d","Type":"ContainerDied","Data":"f8bc8192f714b00a5363924f926b600147bc4b86a1fe23665a0006939e2a7fae"}
Sep 10 21:25:35.345307 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.345261    2573 scope.go:117] "RemoveContainer" containerID="b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e"
Sep 10 21:25:35.355539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.355522    2573 scope.go:117] "RemoveContainer" containerID="4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607"
Sep 10 21:25:35.363960 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.363941    2573 scope.go:117] "RemoveContainer" containerID="b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e"
Sep 10 21:25:35.364230 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:25:35.364200    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e\": container with ID starting with b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e not found: ID does not exist" containerID="b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e"
Sep 10 21:25:35.364287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.364230    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e"} err="failed to get container status \"b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e\": rpc error: code = NotFound desc = could not find container \"b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e\": container with ID starting with b7e997e8db117deff17d489a921b2e0c5b0892851107d163d5e8a887c53cfc8e not found: ID does not exist"
Sep 10 21:25:35.364287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.364249    2573 scope.go:117] "RemoveContainer" containerID="4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607"
Sep 10 21:25:35.364473 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:25:35.364456    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607\": container with ID starting with 4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607 not found: ID does not exist" containerID="4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607"
Sep 10 21:25:35.364512 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.364477    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607"} err="failed to get container status \"4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607\": rpc error: code = NotFound desc = could not find container \"4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607\": container with ID starting with 4c149f7c5ba0aedc30a01a1e18670cb8373f88abac5791c852589579aefee607 not found: ID does not exist"
Sep 10 21:25:35.368281 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.368259    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"]
Sep 10 21:25:35.371697 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:35.371661    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-dc873af1-openshift-default-7b4456f7d6-nfkh8"]
Sep 10 21:25:36.678784 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:36.678741    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" path="/var/lib/kubelet/pods/0632efc6-ccc8-4b43-bb9c-26324dbe400d/volumes"
Sep 10 21:25:39.718716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:39.718672    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:25:46.233054 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:46.233003    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"]
Sep 10 21:25:46.235370 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:46.235331    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy" containerID="cri-o://235c13c77e83491ebdb54478bf705848e38d4039b4ea5847b0e6ef6073b4cfb3" gracePeriod=30
Sep 10 21:25:46.953099 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:46.953066    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"]
Sep 10 21:25:46.953469 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:46.953457    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy"
Sep 10 21:25:46.953516 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:46.953471    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy"
Sep 10 21:25:46.953549 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:46.953538    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy"
Sep 10 21:25:46.956959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:46.956940    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:46.959115 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:46.959093    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-fbe4cf-openshift-default-dockercfg-mk9pq\""
Sep 10 21:25:46.969948 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:46.969918    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"]
Sep 10 21:25:47.047557 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.047514    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581e9ccd-1552-44ce-96c9-5e148f54caa0-istiod-ca-cert\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.047754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.047561    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-credential-socket\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.047754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.047590    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-token\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.047754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.047645    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6f4h\" (UniqueName: \"kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-kube-api-access-n6f4h\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.047923 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.047749    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-data\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.047923 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.047805    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-socket\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.047923 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.047863    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-envoy\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.047923 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.047893    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-podinfo\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.048072 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.047932    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-certs\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.143209 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.143174    2573 patch_prober.go:28] interesting pod/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:25:47.143402 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.143250    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:25:47.148561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.148528    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-envoy\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.148730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.148586    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-podinfo\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.148730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.148647    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-certs\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.148730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.148682    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581e9ccd-1552-44ce-96c9-5e148f54caa0-istiod-ca-cert\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.148730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.148715    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-credential-socket\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.148982 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.148740    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-token\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.148982 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.148798    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6f4h\" (UniqueName: \"kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-kube-api-access-n6f4h\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.148982 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.148874    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-data\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.148982 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.148919    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-socket\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.149226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.149195    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-credential-socket\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.149305 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.149225    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-certs\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.149365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.149313    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-socket\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.149365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.149323    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-data\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.149474 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.149456    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581e9ccd-1552-44ce-96c9-5e148f54caa0-istiod-ca-cert\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.151447 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.151424    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-envoy\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.151768 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.151747    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-podinfo\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.157478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.157453    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6f4h\" (UniqueName: \"kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-kube-api-access-n6f4h\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.158001 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.157980    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-token\") pod \"e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") " pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.268744 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.268650    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:47.566393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.566366    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"]
Sep 10 21:25:47.568408 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:25:47.568374    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581e9ccd_1552_44ce_96c9_5e148f54caa0.slice/crio-c3aad615964921058fc93b8b8de38147414199fb1242dab8ca663dd959e085ce WatchSource:0}: Error finding container c3aad615964921058fc93b8b8de38147414199fb1242dab8ca663dd959e085ce: Status 404 returned error can't find the container with id c3aad615964921058fc93b8b8de38147414199fb1242dab8ca663dd959e085ce
Sep 10 21:25:47.571055 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.571013    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:47.571178 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.571097    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:47.571178 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:47.571147    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:48.425753 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:48.425708    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" event={"ID":"581e9ccd-1552-44ce-96c9-5e148f54caa0","Type":"ContainerStarted","Data":"9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80"}
Sep 10 21:25:48.426113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:48.425792    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" event={"ID":"581e9ccd-1552-44ce-96c9-5e148f54caa0","Type":"ContainerStarted","Data":"c3aad615964921058fc93b8b8de38147414199fb1242dab8ca663dd959e085ce"}
Sep 10 21:25:48.446692 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:48.446634    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" podStartSLOduration=2.446616821 podStartE2EDuration="2.446616821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:48.444014474 +0000 UTC m=+924.377681290" watchObservedRunningTime="2026-09-10 21:25:48.446616821 +0000 UTC m=+924.380283637"
Sep 10 21:25:49.269196 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:49.269151    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:49.274480 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:49.274449    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:49.372002 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:49.371960    2573 patch_prober.go:28] interesting pod/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 21:25:49.372177 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:49.372030    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 21:25:49.431594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:49.431562    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:49.432800 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:49.432761    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:25:51.444940 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.444902    2573 generic.go:356] "Generic (PLEG): container finished" podID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerID="0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b" exitCode=0
Sep 10 21:25:51.445368 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.444969    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" event={"ID":"cf3ef09f-8588-4a82-902a-6407fbb8939f","Type":"ContainerDied","Data":"0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b"}
Sep 10 21:25:51.445478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.445461    2573 scope.go:117] "RemoveContainer" containerID="0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b"
Sep 10 21:25:51.447160 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.447120    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:51.447286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.447225    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:51.447286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.447278    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:25:51.447597 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.447570    2573 generic.go:356] "Generic (PLEG): container finished" podID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerID="235c13c77e83491ebdb54478bf705848e38d4039b4ea5847b0e6ef6073b4cfb3" exitCode=0
Sep 10 21:25:51.447660 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.447641    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" event={"ID":"ac73d580-91be-4437-8103-1b1e70b0fc95","Type":"ContainerDied","Data":"235c13c77e83491ebdb54478bf705848e38d4039b4ea5847b0e6ef6073b4cfb3"}
Sep 10 21:25:51.447716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.447705    2573 scope.go:117] "RemoveContainer" containerID="9bbfe76e3c2224bb9d7957703ac58f9ac36c27fb8f5b7a0d711649c9421ac7f5"
Sep 10 21:25:51.506296 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.506267    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:25:51.596639 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.596605    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-podinfo\") pod \"ac73d580-91be-4437-8103-1b1e70b0fc95\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") "
Sep 10 21:25:51.596857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.596691    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-token\") pod \"ac73d580-91be-4437-8103-1b1e70b0fc95\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") "
Sep 10 21:25:51.596857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.596720    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-envoy\") pod \"ac73d580-91be-4437-8103-1b1e70b0fc95\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") "
Sep 10 21:25:51.596857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.596741    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-data\") pod \"ac73d580-91be-4437-8103-1b1e70b0fc95\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") "
Sep 10 21:25:51.596857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.596762    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac73d580-91be-4437-8103-1b1e70b0fc95-istiod-ca-cert\") pod \"ac73d580-91be-4437-8103-1b1e70b0fc95\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") "
Sep 10 21:25:51.596857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.596818    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdn5w\" (UniqueName: \"kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-kube-api-access-fdn5w\") pod \"ac73d580-91be-4437-8103-1b1e70b0fc95\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") "
Sep 10 21:25:51.596857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.596853    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-credential-socket\") pod \"ac73d580-91be-4437-8103-1b1e70b0fc95\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") "
Sep 10 21:25:51.597220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.596912    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-socket\") pod \"ac73d580-91be-4437-8103-1b1e70b0fc95\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") "
Sep 10 21:25:51.597220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.596943    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-certs\") pod \"ac73d580-91be-4437-8103-1b1e70b0fc95\" (UID: \"ac73d580-91be-4437-8103-1b1e70b0fc95\") "
Sep 10 21:25:51.598545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.597631    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-data" (OuterVolumeSpecName: "istio-data") pod "ac73d580-91be-4437-8103-1b1e70b0fc95" (UID: "ac73d580-91be-4437-8103-1b1e70b0fc95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:51.598545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.597900    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:51.598545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.598017    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ac73d580-91be-4437-8103-1b1e70b0fc95" (UID: "ac73d580-91be-4437-8103-1b1e70b0fc95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:51.598545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.598367    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ac73d580-91be-4437-8103-1b1e70b0fc95" (UID: "ac73d580-91be-4437-8103-1b1e70b0fc95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:51.598879 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.598665    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ac73d580-91be-4437-8103-1b1e70b0fc95" (UID: "ac73d580-91be-4437-8103-1b1e70b0fc95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:51.599124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.599089    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac73d580-91be-4437-8103-1b1e70b0fc95-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ac73d580-91be-4437-8103-1b1e70b0fc95" (UID: "ac73d580-91be-4437-8103-1b1e70b0fc95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:51.599727 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.599705    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ac73d580-91be-4437-8103-1b1e70b0fc95" (UID: "ac73d580-91be-4437-8103-1b1e70b0fc95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:51.599897 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.599877    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ac73d580-91be-4437-8103-1b1e70b0fc95" (UID: "ac73d580-91be-4437-8103-1b1e70b0fc95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:25:51.600994 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.600959    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-token" (OuterVolumeSpecName: "istio-token") pod "ac73d580-91be-4437-8103-1b1e70b0fc95" (UID: "ac73d580-91be-4437-8103-1b1e70b0fc95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:51.601951 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.601919    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-kube-api-access-fdn5w" (OuterVolumeSpecName: "kube-api-access-fdn5w") pod "ac73d580-91be-4437-8103-1b1e70b0fc95" (UID: "ac73d580-91be-4437-8103-1b1e70b0fc95"). InnerVolumeSpecName "kube-api-access-fdn5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:51.699123 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.699081    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:51.699123 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.699123    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:51.699376 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.699138    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:51.699376 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.699151    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:51.699376 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.699164    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:51.699376 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.699175    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac73d580-91be-4437-8103-1b1e70b0fc95-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:51.699376 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.699186    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdn5w\" (UniqueName: \"kubernetes.io/projected/ac73d580-91be-4437-8103-1b1e70b0fc95-kube-api-access-fdn5w\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:51.699376 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:51.699199    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73d580-91be-4437-8103-1b1e70b0fc95-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:25:52.455701 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:52.455665    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" event={"ID":"cf3ef09f-8588-4a82-902a-6407fbb8939f","Type":"ContainerStarted","Data":"b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c"}
Sep 10 21:25:52.457107 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:52.457088    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"
Sep 10 21:25:52.457227 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:52.457089    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k" event={"ID":"ac73d580-91be-4437-8103-1b1e70b0fc95","Type":"ContainerDied","Data":"8c4c94fc10d7d1407b8959dcc271109285855e2dcf411232d342f8fb1013083d"}
Sep 10 21:25:52.457296 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:52.457225    2573 scope.go:117] "RemoveContainer" containerID="235c13c77e83491ebdb54478bf705848e38d4039b4ea5847b0e6ef6073b4cfb3"
Sep 10 21:25:52.496589 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:52.496546    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"]
Sep 10 21:25:52.501631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:52.501601    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-f135f4-openshift-default-7657c96bb8-njv5k"]
Sep 10 21:25:52.684378 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:52.684331    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" path="/var/lib/kubelet/pods/ac73d580-91be-4437-8103-1b1e70b0fc95/volumes"
Sep 10 21:25:52.749652 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:52.749564    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:52.749833 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:52.749794    2573 patch_prober.go:28] interesting pod/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:25:52.749886 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:52.749854    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 10 21:25:53.749870 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:53.749830    2573 patch_prober.go:28] interesting pod/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:25:53.750271 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:53.749889    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 10 21:25:54.750250 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:54.750215    2573 patch_prober.go:28] interesting pod/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:25:54.750667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:54.750273    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 10 21:25:55.749622 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:55.749583    2573 patch_prober.go:28] interesting pod/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:25:55.749822 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:55.749640    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 10 21:25:56.749676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:56.749639    2573 patch_prober.go:28] interesting pod/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:25:56.750132 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:56.749701    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 10 21:25:57.477260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.477212    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"]
Sep 10 21:25:57.477811 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.477768    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy"
Sep 10 21:25:57.477917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.477814    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy"
Sep 10 21:25:57.477917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.477834    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy"
Sep 10 21:25:57.477917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.477843    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy"
Sep 10 21:25:57.478097 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.477989    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0632efc6-ccc8-4b43-bb9c-26324dbe400d" containerName="istio-proxy"
Sep 10 21:25:57.478097 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.478005    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy"
Sep 10 21:25:57.481879 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.481851    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"
Sep 10 21:25:57.484703 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.484680    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:25:57.489752 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.489715    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"]
Sep 10 21:25:57.558135 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.558085    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2f3efd4-3016-4bd3-9a8c-65c5541c3665-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fd195260-2wx7t\" (UID: \"b2f3efd4-3016-4bd3-9a8c-65c5541c3665\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"
Sep 10 21:25:57.658868 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.658822    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2f3efd4-3016-4bd3-9a8c-65c5541c3665-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fd195260-2wx7t\" (UID: \"b2f3efd4-3016-4bd3-9a8c-65c5541c3665\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"
Sep 10 21:25:57.659488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.659464    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2f3efd4-3016-4bd3-9a8c-65c5541c3665-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fd195260-2wx7t\" (UID: \"b2f3efd4-3016-4bd3-9a8c-65c5541c3665\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"
Sep 10 21:25:57.749584 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.749489    2573 patch_prober.go:28] interesting pod/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:25:57.749584 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.749546    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 10 21:25:57.807861 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:57.807817    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"
Sep 10 21:25:58.083262 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:58.083219    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"]
Sep 10 21:25:58.083708 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:25:58.083675    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2f3efd4_3016_4bd3_9a8c_65c5541c3665.slice/crio-20cce115dc1d8d18112826af18cf5a812ebbabf59048c879873788072ee30942 WatchSource:0}: Error finding container 20cce115dc1d8d18112826af18cf5a812ebbabf59048c879873788072ee30942: Status 404 returned error can't find the container with id 20cce115dc1d8d18112826af18cf5a812ebbabf59048c879873788072ee30942
Sep 10 21:25:58.315506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:58.315459    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:25:58.513378 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:58.513347    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t" event={"ID":"b2f3efd4-3016-4bd3-9a8c-65c5541c3665","Type":"ContainerStarted","Data":"b4c3c0af66ee1f5a0daba1bef6337be74101a4a231e77019cc3ae7ef68645d7d"}
Sep 10 21:25:58.513537 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:58.513398    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t" event={"ID":"b2f3efd4-3016-4bd3-9a8c-65c5541c3665","Type":"ContainerStarted","Data":"20cce115dc1d8d18112826af18cf5a812ebbabf59048c879873788072ee30942"}
Sep 10 21:25:58.528980 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:58.528919    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t" podStartSLOduration=1.528902407 podStartE2EDuration="1.528902407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:58.527067351 +0000 UTC m=+934.460734168" watchObservedRunningTime="2026-09-10 21:25:58.528902407 +0000 UTC m=+934.462569221"
Sep 10 21:25:58.754005 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:58.753876    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:58.754373 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:58.754268    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:58.754929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:58.754907    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:25:59.523495 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:59.523464    2573 generic.go:356] "Generic (PLEG): container finished" podID="b2f3efd4-3016-4bd3-9a8c-65c5541c3665" containerID="b4c3c0af66ee1f5a0daba1bef6337be74101a4a231e77019cc3ae7ef68645d7d" exitCode=0
Sep 10 21:25:59.523648 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:25:59.523549    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t" event={"ID":"b2f3efd4-3016-4bd3-9a8c-65c5541c3665","Type":"ContainerDied","Data":"b4c3c0af66ee1f5a0daba1bef6337be74101a4a231e77019cc3ae7ef68645d7d"}
Sep 10 21:26:00.101474 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.101426    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"]
Sep 10 21:26:00.102324 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.102297    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy"
Sep 10 21:26:00.102324 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.102326    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy"
Sep 10 21:26:00.102544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.102530    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac73d580-91be-4437-8103-1b1e70b0fc95" containerName="istio-proxy"
Sep 10 21:26:00.105551 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.105527    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.108103 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.108083    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-788af409-kserve-self-signed-certs\""
Sep 10 21:26:00.111565 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.111541    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"]
Sep 10 21:26:00.191801 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.188134    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tls-certs\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.191801 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.188191    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzl4d\" (UniqueName: \"kubernetes.io/projected/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-kube-api-access-vzl4d\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.191801 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.188249    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tmp-dir\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.191801 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.188374    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-dshm\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.191801 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.188406    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-model-cache\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.191801 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.188470    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-home\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.289805 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.289748    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-dshm\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.290080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.290054    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-model-cache\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.290249 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.290232    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-home\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.290473 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.290457    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tls-certs\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.290596 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.290579    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzl4d\" (UniqueName: \"kubernetes.io/projected/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-kube-api-access-vzl4d\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.290717 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.290702    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tmp-dir\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.290906 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.290588    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-home\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.290906 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.290498    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-model-cache\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.291172 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.291144    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tmp-dir\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.295113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.295082    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-dshm\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.295379 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.295353    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tls-certs\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.298953 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.298924    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzl4d\" (UniqueName: \"kubernetes.io/projected/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-kube-api-access-vzl4d\") pod \"e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") " pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.416339 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.416245    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:00.703113 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.703092    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"
Sep 10 21:26:00.704190 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.704162    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"]
Sep 10 21:26:00.705958 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:26:00.705937    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4f1c1d0_7ae5_44b8_a481_ede82202c1f7.slice/crio-3f179cb6dbd420ca026a9ba8d1d4dad93597dce7b468fdd6b0d33f2644d9f7b4 WatchSource:0}: Error finding container 3f179cb6dbd420ca026a9ba8d1d4dad93597dce7b468fdd6b0d33f2644d9f7b4: Status 404 returned error can't find the container with id 3f179cb6dbd420ca026a9ba8d1d4dad93597dce7b468fdd6b0d33f2644d9f7b4
Sep 10 21:26:00.800702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.800669    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2f3efd4-3016-4bd3-9a8c-65c5541c3665-maas-api-service-ca\") pod \"b2f3efd4-3016-4bd3-9a8c-65c5541c3665\" (UID: \"b2f3efd4-3016-4bd3-9a8c-65c5541c3665\") "
Sep 10 21:26:00.801018 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.800994    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2f3efd4-3016-4bd3-9a8c-65c5541c3665-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b2f3efd4-3016-4bd3-9a8c-65c5541c3665" (UID: "b2f3efd4-3016-4bd3-9a8c-65c5541c3665"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:00.902168 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:00.902131    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2f3efd4-3016-4bd3-9a8c-65c5541c3665-maas-api-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:01.537102 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:01.537072    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"
Sep 10 21:26:01.537102 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:01.537092    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t" event={"ID":"b2f3efd4-3016-4bd3-9a8c-65c5541c3665","Type":"ContainerDied","Data":"20cce115dc1d8d18112826af18cf5a812ebbabf59048c879873788072ee30942"}
Sep 10 21:26:01.537647 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:01.537143    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20cce115dc1d8d18112826af18cf5a812ebbabf59048c879873788072ee30942"
Sep 10 21:26:01.538882 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:01.538856    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb" event={"ID":"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7","Type":"ContainerStarted","Data":"8a7f9532589bb38b21af8e7ad5a0a3cb191d7f617e85fd6f00590c9ff6ac2406"}
Sep 10 21:26:01.538998 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:01.538889    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb" event={"ID":"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7","Type":"ContainerStarted","Data":"3f179cb6dbd420ca026a9ba8d1d4dad93597dce7b468fdd6b0d33f2644d9f7b4"}
Sep 10 21:26:01.557446 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:01.557399    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb" podStartSLOduration=1.557383061 podStartE2EDuration="1.557383061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:26:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:26:01.556078615 +0000 UTC m=+937.489745422" watchObservedRunningTime="2026-09-10 21:26:01.557383061 +0000 UTC m=+937.491049921"
Sep 10 21:26:04.820708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:04.820668    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"]
Sep 10 21:26:04.822984 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:04.822963    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fd195260-2wx7t"]
Sep 10 21:26:04.954285 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:04.954233    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"]
Sep 10 21:26:04.954605 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:04.954575    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy" containerID="cri-o://b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c" gracePeriod=30
Sep 10 21:26:06.680454 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:06.680415    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2f3efd4-3016-4bd3-9a8c-65c5541c3665" path="/var/lib/kubelet/pods/b2f3efd4-3016-4bd3-9a8c-65c5541c3665/volumes"
Sep 10 21:26:08.582050 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:08.582022    2573 generic.go:356] "Generic (PLEG): container finished" podID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerID="9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80" exitCode=0
Sep 10 21:26:08.582484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:08.582096    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" event={"ID":"581e9ccd-1552-44ce-96c9-5e148f54caa0","Type":"ContainerDied","Data":"9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80"}
Sep 10 21:26:08.582484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:08.582470    2573 scope.go:117] "RemoveContainer" containerID="9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80"
Sep 10 21:26:08.583491 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:08.583466    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:26:08.583566 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:08.583536    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:26:08.583607 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:08.583567    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:26:09.594270 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:09.594228    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" event={"ID":"581e9ccd-1552-44ce-96c9-5e148f54caa0","Type":"ContainerStarted","Data":"a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e"}
Sep 10 21:26:10.242854 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.242825    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:26:10.269178 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.269089    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:26:10.269343 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.269320    2573 patch_prober.go:28] interesting pod/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:26:10.269399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.269375    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 10 21:26:10.298462 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.298422    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-socket\") pod \"cf3ef09f-8588-4a82-902a-6407fbb8939f\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") "
Sep 10 21:26:10.298702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.298496    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-token\") pod \"cf3ef09f-8588-4a82-902a-6407fbb8939f\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") "
Sep 10 21:26:10.298702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.298551    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cvmq\" (UniqueName: \"kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-kube-api-access-2cvmq\") pod \"cf3ef09f-8588-4a82-902a-6407fbb8939f\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") "
Sep 10 21:26:10.298702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.298597    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-envoy\") pod \"cf3ef09f-8588-4a82-902a-6407fbb8939f\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") "
Sep 10 21:26:10.298928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.298727    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf3ef09f-8588-4a82-902a-6407fbb8939f" (UID: "cf3ef09f-8588-4a82-902a-6407fbb8939f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:10.298928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.298770    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-podinfo\") pod \"cf3ef09f-8588-4a82-902a-6407fbb8939f\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") "
Sep 10 21:26:10.298928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.298823    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-credential-socket\") pod \"cf3ef09f-8588-4a82-902a-6407fbb8939f\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") "
Sep 10 21:26:10.298928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.298855    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-data\") pod \"cf3ef09f-8588-4a82-902a-6407fbb8939f\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") "
Sep 10 21:26:10.298928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.298881    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-certs\") pod \"cf3ef09f-8588-4a82-902a-6407fbb8939f\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") "
Sep 10 21:26:10.298928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.298906    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf3ef09f-8588-4a82-902a-6407fbb8939f-istiod-ca-cert\") pod \"cf3ef09f-8588-4a82-902a-6407fbb8939f\" (UID: \"cf3ef09f-8588-4a82-902a-6407fbb8939f\") "
Sep 10 21:26:10.299232 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.299159    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf3ef09f-8588-4a82-902a-6407fbb8939f" (UID: "cf3ef09f-8588-4a82-902a-6407fbb8939f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:10.299297 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.299247    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf3ef09f-8588-4a82-902a-6407fbb8939f" (UID: "cf3ef09f-8588-4a82-902a-6407fbb8939f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:10.299356 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.299305    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:10.299356 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.299326    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:10.299457 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.299367    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf3ef09f-8588-4a82-902a-6407fbb8939f" (UID: "cf3ef09f-8588-4a82-902a-6407fbb8939f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:10.299533 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.299511    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf3ef09f-8588-4a82-902a-6407fbb8939f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf3ef09f-8588-4a82-902a-6407fbb8939f" (UID: "cf3ef09f-8588-4a82-902a-6407fbb8939f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:10.301368 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.301329    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-kube-api-access-2cvmq" (OuterVolumeSpecName: "kube-api-access-2cvmq") pod "cf3ef09f-8588-4a82-902a-6407fbb8939f" (UID: "cf3ef09f-8588-4a82-902a-6407fbb8939f"). InnerVolumeSpecName "kube-api-access-2cvmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:10.301570 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.301539    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf3ef09f-8588-4a82-902a-6407fbb8939f" (UID: "cf3ef09f-8588-4a82-902a-6407fbb8939f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:26:10.301753 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.301726    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf3ef09f-8588-4a82-902a-6407fbb8939f" (UID: "cf3ef09f-8588-4a82-902a-6407fbb8939f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:10.301997 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.301971    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf3ef09f-8588-4a82-902a-6407fbb8939f" (UID: "cf3ef09f-8588-4a82-902a-6407fbb8939f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:10.400480 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.400442    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:10.400480 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.400471    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cvmq\" (UniqueName: \"kubernetes.io/projected/cf3ef09f-8588-4a82-902a-6407fbb8939f-kube-api-access-2cvmq\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:10.400480 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.400482    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:10.400480 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.400491    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf3ef09f-8588-4a82-902a-6407fbb8939f-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:10.400752 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.400499    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:10.400752 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.400506    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf3ef09f-8588-4a82-902a-6407fbb8939f-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:10.400752 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.400514    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf3ef09f-8588-4a82-902a-6407fbb8939f-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:10.417107 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.417057    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:10.417272 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.417125    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:10.429843 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.429814    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:10.601797 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.601744    2573 generic.go:356] "Generic (PLEG): container finished" podID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerID="b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c" exitCode=0
Sep 10 21:26:10.602252 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.601806    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" event={"ID":"cf3ef09f-8588-4a82-902a-6407fbb8939f","Type":"ContainerDied","Data":"b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c"}
Sep 10 21:26:10.602252 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.601819    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"
Sep 10 21:26:10.602252 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.601857    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm" event={"ID":"cf3ef09f-8588-4a82-902a-6407fbb8939f","Type":"ContainerDied","Data":"9df1508bebf03f87156d10890a8f4d877eed22b5a18d6d0302e136b32c405b3e"}
Sep 10 21:26:10.602252 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.601879    2573 scope.go:117] "RemoveContainer" containerID="b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c"
Sep 10 21:26:10.613243 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.613227    2573 scope.go:117] "RemoveContainer" containerID="0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b"
Sep 10 21:26:10.615984 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.615959    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:10.621978 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.621961    2573 scope.go:117] "RemoveContainer" containerID="b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c"
Sep 10 21:26:10.622288 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:26:10.622263    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c\": container with ID starting with b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c not found: ID does not exist" containerID="b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c"
Sep 10 21:26:10.622380 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.622300    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c"} err="failed to get container status \"b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c\": rpc error: code = NotFound desc = could not find container \"b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c\": container with ID starting with b44e95ac26bfe673f2f4250959157873be01df86640b3d990b0780e9eb7efa6c not found: ID does not exist"
Sep 10 21:26:10.622380 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.622323    2573 scope.go:117] "RemoveContainer" containerID="0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b"
Sep 10 21:26:10.622615 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:26:10.622598    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b\": container with ID starting with 0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b not found: ID does not exist" containerID="0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b"
Sep 10 21:26:10.622668 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.622620    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b"} err="failed to get container status \"0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b\": rpc error: code = NotFound desc = could not find container \"0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b\": container with ID starting with 0a4c0a73ae276ca215693214d78f04d386ba99f973f2abaaec3866679fe29d3b not found: ID does not exist"
Sep 10 21:26:10.624533 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.624515    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"]
Sep 10 21:26:10.628245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.628221    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-fd195260-openshift-default-66c948d69b-gt5bm"]
Sep 10 21:26:10.701675 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:10.701623    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" path="/var/lib/kubelet/pods/cf3ef09f-8588-4a82-902a-6407fbb8939f/volumes"
Sep 10 21:26:11.269124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:11.269089    2573 patch_prober.go:28] interesting pod/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:26:11.269308 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:11.269149    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 10 21:26:12.109789 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:12.109728    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:26:12.269226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:12.269195    2573 patch_prober.go:28] interesting pod/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:26:12.269408 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:12.269255    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 10 21:26:13.269699 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:13.269661    2573 patch_prober.go:28] interesting pod/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:26:13.270128 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:13.269734    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 10 21:26:14.269303 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:14.269270    2573 patch_prober.go:28] interesting pod/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:26:14.269484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:14.269337    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 10 21:26:15.273327 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:15.273294    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:26:15.273723 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:15.273545    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:26:15.274220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:15.274202    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:26:17.591089 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:17.591055    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"]
Sep 10 21:26:23.627665 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:23.622808    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:26:25.511060 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.511024    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"]
Sep 10 21:26:25.511435 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.511419    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2f3efd4-3016-4bd3-9a8c-65c5541c3665" containerName="revoke-keys"
Sep 10 21:26:25.511478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.511437    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2f3efd4-3016-4bd3-9a8c-65c5541c3665" containerName="revoke-keys"
Sep 10 21:26:25.511478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.511446    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy"
Sep 10 21:26:25.511478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.511451    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy"
Sep 10 21:26:25.511478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.511469    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy"
Sep 10 21:26:25.511478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.511474    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy"
Sep 10 21:26:25.511631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.511547    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy"
Sep 10 21:26:25.511631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.511558    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf3ef09f-8588-4a82-902a-6407fbb8939f" containerName="istio-proxy"
Sep 10 21:26:25.511631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.511563    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2f3efd4-3016-4bd3-9a8c-65c5541c3665" containerName="revoke-keys"
Sep 10 21:26:25.515176 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.515155    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.517753 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.517729    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-fbe4cf\"/\"kube-root-ca.crt\""
Sep 10 21:26:25.517899 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.517816    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-fbe4cf\"/\"openshift-service-ca.crt\""
Sep 10 21:26:25.517899 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.517828    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-fbe4cf\"/\"explicit-model-fbe4cf-kserve-self-signed-certs\""
Sep 10 21:26:25.517899 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.517849    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-fbe4cf\"/\"default-dockercfg-ptvw5\""
Sep 10 21:26:25.523641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.523619    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"]
Sep 10 21:26:25.646249 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.646219    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-model-cache\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.646421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.646258    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tmp-dir\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.646421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.646276    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-dshm\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.646421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.646329    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfcx9\" (UniqueName: \"kubernetes.io/projected/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-kube-api-access-xfcx9\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.646421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.646381    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tls-certs\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.646421 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.646411    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-home\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.747066 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.747031    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfcx9\" (UniqueName: \"kubernetes.io/projected/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-kube-api-access-xfcx9\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.747261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.747076    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tls-certs\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.747261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.747101    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-home\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.747261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.747154    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-model-cache\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.747261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.747187    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tmp-dir\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.747261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.747210    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-dshm\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.747567 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.747549    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-home\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.747610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.747595    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tmp-dir\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.747664 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.747637    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-model-cache\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.749452 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.749427    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-dshm\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.749744 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.749724    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tls-certs\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.755274 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.755255    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfcx9\" (UniqueName: \"kubernetes.io/projected/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-kube-api-access-xfcx9\") pod \"explicit-model-fbe4cf-kserve-85899c7fc-z2mwf\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") " pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:25.825331 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:25.825296    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:26.097474 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:26.097448    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"]
Sep 10 21:26:26.099226 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:26:26.099184    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07bbcdbf_d7c9_451e_9348_47538f4ba5ac.slice/crio-9140aa34ba163ab7250b3fada5872e08e3cd7b86365b543a9507fd82b9bfc222 WatchSource:0}: Error finding container 9140aa34ba163ab7250b3fada5872e08e3cd7b86365b543a9507fd82b9bfc222: Status 404 returned error can't find the container with id 9140aa34ba163ab7250b3fada5872e08e3cd7b86365b543a9507fd82b9bfc222
Sep 10 21:26:26.693697 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:26.693654    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf" event={"ID":"07bbcdbf-d7c9-451e-9348-47538f4ba5ac","Type":"ContainerStarted","Data":"9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231"}
Sep 10 21:26:26.693697 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:26.693697    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf" event={"ID":"07bbcdbf-d7c9-451e-9348-47538f4ba5ac","Type":"ContainerStarted","Data":"9140aa34ba163ab7250b3fada5872e08e3cd7b86365b543a9507fd82b9bfc222"}
Sep 10 21:26:26.710803 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:26.710736    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf" podStartSLOduration=1.710709494 podStartE2EDuration="1.710709494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:26:26.708207198 +0000 UTC m=+962.641874012" watchObservedRunningTime="2026-09-10 21:26:26.710709494 +0000 UTC m=+962.644376300"
Sep 10 21:26:32.611898 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:32.611841    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb" podUID="e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" containerName="main" containerID="cri-o://8a7f9532589bb38b21af8e7ad5a0a3cb191d7f617e85fd6f00590c9ff6ac2406" gracePeriod=45
Sep 10 21:26:32.729434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:32.729405    2573 generic.go:356] "Generic (PLEG): container finished" podID="e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" containerID="8a7f9532589bb38b21af8e7ad5a0a3cb191d7f617e85fd6f00590c9ff6ac2406" exitCode=0
Sep 10 21:26:32.729545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:32.729474    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb" event={"ID":"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7","Type":"ContainerDied","Data":"8a7f9532589bb38b21af8e7ad5a0a3cb191d7f617e85fd6f00590c9ff6ac2406"}
Sep 10 21:26:32.868705 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:32.868648    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:33.017656 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.017621    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-home\") pod \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") "
Sep 10 21:26:33.017858 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.017669    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-dshm\") pod \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") "
Sep 10 21:26:33.017858 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.017702    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-model-cache\") pod \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") "
Sep 10 21:26:33.017858 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.017756    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tmp-dir\") pod \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") "
Sep 10 21:26:33.017858 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.017802    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tls-certs\") pod \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") "
Sep 10 21:26:33.017858 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.017830    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzl4d\" (UniqueName: \"kubernetes.io/projected/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-kube-api-access-vzl4d\") pod \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\" (UID: \"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7\") "
Sep 10 21:26:33.018146 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.017926    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-home" (OuterVolumeSpecName: "home") pod "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" (UID: "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:33.018146 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.018034    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" (UID: "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:33.018146 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.018047    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-model-cache" (OuterVolumeSpecName: "model-cache") pod "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" (UID: "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:33.018256 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.018181    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:33.018256 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.018194    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:33.018256 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.018205    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:33.020093 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.020058    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-dshm" (OuterVolumeSpecName: "dshm") pod "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" (UID: "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:33.020219 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.020149    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" (UID: "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:33.020219 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.020171    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-kube-api-access-vzl4d" (OuterVolumeSpecName: "kube-api-access-vzl4d") pod "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" (UID: "e4f1c1d0-7ae5-44b8-a481-ede82202c1f7"). InnerVolumeSpecName "kube-api-access-vzl4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:33.118900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.118810    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:33.118900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.118841    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:33.118900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.118853    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzl4d\" (UniqueName: \"kubernetes.io/projected/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7-kube-api-access-vzl4d\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:33.741832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.741800    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"
Sep 10 21:26:33.741832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.741819    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb" event={"ID":"e4f1c1d0-7ae5-44b8-a481-ede82202c1f7","Type":"ContainerDied","Data":"3f179cb6dbd420ca026a9ba8d1d4dad93597dce7b468fdd6b0d33f2644d9f7b4"}
Sep 10 21:26:33.742380 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.741870    2573 scope.go:117] "RemoveContainer" containerID="8a7f9532589bb38b21af8e7ad5a0a3cb191d7f617e85fd6f00590c9ff6ac2406"
Sep 10 21:26:33.763881 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.763844    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"]
Sep 10 21:26:33.769488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:33.769464    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-788af409-kserve-dc84b57b8-h4zcb"]
Sep 10 21:26:34.678749 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:34.678717    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" path="/var/lib/kubelet/pods/e4f1c1d0-7ae5-44b8-a481-ede82202c1f7/volumes"
Sep 10 21:26:35.805765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:35.805721    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:26:35.826295 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:35.826268    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:35.826451 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:35.826309    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:35.838872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:35.838846    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:36.770058 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:36.770027    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:40.223066 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.223031    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"]
Sep 10 21:26:40.223472 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.223432    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" containerName="main"
Sep 10 21:26:40.223472 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.223444    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" containerName="main"
Sep 10 21:26:40.223562 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.223505    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4f1c1d0-7ae5-44b8-a481-ede82202c1f7" containerName="main"
Sep 10 21:26:40.226382 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.226366    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.229101 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.229077    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a6441b\"/\"test-model-a6441b-kserve-self-signed-certs\""
Sep 10 21:26:40.229249 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.229128    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a6441b\"/\"openshift-service-ca.crt\""
Sep 10 21:26:40.229249 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.229217    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a6441b\"/\"default-dockercfg-cjnss\""
Sep 10 21:26:40.229249 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.229234    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a6441b\"/\"kube-root-ca.crt\""
Sep 10 21:26:40.234464 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.234434    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"]
Sep 10 21:26:40.389064 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.389029    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmxh8\" (UniqueName: \"kubernetes.io/projected/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-kube-api-access-qmxh8\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.389244 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.389091    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-dshm\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.389244 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.389186    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-model-cache\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.389244 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.389224    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-home\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.389357 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.389257    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tls-certs\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.389357 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.389329    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tmp-dir\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.490329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.490210    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tls-certs\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.490329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.490258    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tmp-dir\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.490329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.490312    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmxh8\" (UniqueName: \"kubernetes.io/projected/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-kube-api-access-qmxh8\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.490795 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.490368    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-dshm\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.490795 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.490419    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-model-cache\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.490795 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.490445    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-home\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.491042 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.490809    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tmp-dir\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.491042 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.490822    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-home\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.491184 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.491112    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-model-cache\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.493092 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.493067    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-dshm\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.493531 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.493499    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tls-certs\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.497757 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.497736    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmxh8\" (UniqueName: \"kubernetes.io/projected/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-kube-api-access-qmxh8\") pod \"test-model-a6441b-kserve-5569dd99f7-j6qp8\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") " pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.536785 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.536742    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:40.805176 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:40.805152    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"]
Sep 10 21:26:40.806327 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:26:40.806291    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6780bcb1_34ff_4dd3_81d1_1888b9d1779b.slice/crio-5e8dd7867ef69550b44680ce309a919fc2fe8292b9070ae1322a66905e8a409c WatchSource:0}: Error finding container 5e8dd7867ef69550b44680ce309a919fc2fe8292b9070ae1322a66905e8a409c: Status 404 returned error can't find the container with id 5e8dd7867ef69550b44680ce309a919fc2fe8292b9070ae1322a66905e8a409c
Sep 10 21:26:41.788424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:41.788381    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8" event={"ID":"6780bcb1-34ff-4dd3-81d1-1888b9d1779b","Type":"ContainerStarted","Data":"e92d98366ffdf6829288cfe8280c3069c6d2c47990df0be2ad98a3d2b96b0bf0"}
Sep 10 21:26:41.788915 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:41.788430    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8" event={"ID":"6780bcb1-34ff-4dd3-81d1-1888b9d1779b","Type":"ContainerStarted","Data":"5e8dd7867ef69550b44680ce309a919fc2fe8292b9070ae1322a66905e8a409c"}
Sep 10 21:26:41.804386 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:41.804332    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8" podStartSLOduration=1.804313128 podStartE2EDuration="1.804313128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:26:41.803556026 +0000 UTC m=+977.737222842" watchObservedRunningTime="2026-09-10 21:26:41.804313128 +0000 UTC m=+977.737979944"
Sep 10 21:26:44.446685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:44.446651    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"]
Sep 10 21:26:49.220356 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:49.220260    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:26:50.537229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:50.537189    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:50.537651 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:50.537250    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:50.551158 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:50.551128    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:50.858210 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:50.858174    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:26:59.465957 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.465914    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf" podUID="07bbcdbf-d7c9-451e-9348-47538f4ba5ac" containerName="main" containerID="cri-o://9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231" gracePeriod=45
Sep 10 21:26:59.719365 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.719305    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:59.777935 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.777894    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-dshm\") pod \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") "
Sep 10 21:26:59.778109 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.777941    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfcx9\" (UniqueName: \"kubernetes.io/projected/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-kube-api-access-xfcx9\") pod \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") "
Sep 10 21:26:59.778109 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.777976    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-model-cache\") pod \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") "
Sep 10 21:26:59.778109 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.778040    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tmp-dir\") pod \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") "
Sep 10 21:26:59.778109 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.778089    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-home\") pod \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") "
Sep 10 21:26:59.778332 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.778163    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tls-certs\") pod \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\" (UID: \"07bbcdbf-d7c9-451e-9348-47538f4ba5ac\") "
Sep 10 21:26:59.778332 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.778283    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "07bbcdbf-d7c9-451e-9348-47538f4ba5ac" (UID: "07bbcdbf-d7c9-451e-9348-47538f4ba5ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:59.778449 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.778370    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07bbcdbf-d7c9-451e-9348-47538f4ba5ac" (UID: "07bbcdbf-d7c9-451e-9348-47538f4ba5ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:59.778449 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.778383    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-home" (OuterVolumeSpecName: "home") pod "07bbcdbf-d7c9-451e-9348-47538f4ba5ac" (UID: "07bbcdbf-d7c9-451e-9348-47538f4ba5ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:59.778572 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.778555    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:59.778627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.778577    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:59.778627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.778589    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:59.780551 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.780523    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-dshm" (OuterVolumeSpecName: "dshm") pod "07bbcdbf-d7c9-451e-9348-47538f4ba5ac" (UID: "07bbcdbf-d7c9-451e-9348-47538f4ba5ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:59.780705 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.780626    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07bbcdbf-d7c9-451e-9348-47538f4ba5ac" (UID: "07bbcdbf-d7c9-451e-9348-47538f4ba5ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:59.780705 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.780638    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-kube-api-access-xfcx9" (OuterVolumeSpecName: "kube-api-access-xfcx9") pod "07bbcdbf-d7c9-451e-9348-47538f4ba5ac" (UID: "07bbcdbf-d7c9-451e-9348-47538f4ba5ac"). InnerVolumeSpecName "kube-api-access-xfcx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:59.879545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.879510    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfcx9\" (UniqueName: \"kubernetes.io/projected/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-kube-api-access-xfcx9\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:59.879545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.879542    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:59.879545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.879553    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07bbcdbf-d7c9-451e-9348-47538f4ba5ac-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:26:59.890409 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.890377    2573 generic.go:356] "Generic (PLEG): container finished" podID="07bbcdbf-d7c9-451e-9348-47538f4ba5ac" containerID="9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231" exitCode=0
Sep 10 21:26:59.890542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.890422    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"
Sep 10 21:26:59.890542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.890437    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf" event={"ID":"07bbcdbf-d7c9-451e-9348-47538f4ba5ac","Type":"ContainerDied","Data":"9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231"}
Sep 10 21:26:59.890542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.890475    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf" event={"ID":"07bbcdbf-d7c9-451e-9348-47538f4ba5ac","Type":"ContainerDied","Data":"9140aa34ba163ab7250b3fada5872e08e3cd7b86365b543a9507fd82b9bfc222"}
Sep 10 21:26:59.890542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.890497    2573 scope.go:117] "RemoveContainer" containerID="9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231"
Sep 10 21:26:59.900402 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.900386    2573 scope.go:117] "RemoveContainer" containerID="9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231"
Sep 10 21:26:59.900661 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:26:59.900642    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231\": container with ID starting with 9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231 not found: ID does not exist" containerID="9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231"
Sep 10 21:26:59.900703 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.900672    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231"} err="failed to get container status \"9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231\": rpc error: code = NotFound desc = could not find container \"9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231\": container with ID starting with 9e97291c9caa2badd9a3c55900affe4158a1467f78e2f27ba61fd141054c8231 not found: ID does not exist"
Sep 10 21:26:59.919418 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.919391    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"]
Sep 10 21:26:59.921919 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:26:59.921896    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-fbe4cf/explicit-model-fbe4cf-kserve-85899c7fc-z2mwf"]
Sep 10 21:27:00.679209 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:00.679181    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07bbcdbf-d7c9-451e-9348-47538f4ba5ac" path="/var/lib/kubelet/pods/07bbcdbf-d7c9-451e-9348-47538f4ba5ac/volumes"
Sep 10 21:27:02.445415 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:02.445383    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"]
Sep 10 21:27:02.445897 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:02.445653    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy" containerID="cri-o://a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e" gracePeriod=30
Sep 10 21:27:03.200286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.200253    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"]
Sep 10 21:27:03.200663 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.200650    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07bbcdbf-d7c9-451e-9348-47538f4ba5ac" containerName="main"
Sep 10 21:27:03.200709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.200664    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="07bbcdbf-d7c9-451e-9348-47538f4ba5ac" containerName="main"
Sep 10 21:27:03.200742 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.200737    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="07bbcdbf-d7c9-451e-9348-47538f4ba5ac" containerName="main"
Sep 10 21:27:03.203555 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.203539    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.205894 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.205875    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-8e4b08-openshift-default-dockercfg-7drml\""
Sep 10 21:27:03.217545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.217518    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"]
Sep 10 21:27:03.314240 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.314201    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-token\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.314240 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.314243    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-socket\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.314455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.314278    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m4gl\" (UniqueName: \"kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-kube-api-access-2m4gl\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.314455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.314298    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-certs\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.314455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.314322    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istiod-ca-cert\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.314455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.314370    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-credential-socket\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.314455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.314403    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-data\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.314614 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.314466    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-envoy\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.314614 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.314483    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-podinfo\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415235 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415199    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m4gl\" (UniqueName: \"kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-kube-api-access-2m4gl\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415235 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415238    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-certs\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415265    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istiod-ca-cert\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415292    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-credential-socket\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415316    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-data\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415353    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-envoy\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415377    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-podinfo\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415471    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-token\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415500    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-socket\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415958 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415687    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-credential-socket\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415958 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415704    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-certs\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.415958 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415872    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-data\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.416082 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.415988    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-socket\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.416131 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.416114    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istiod-ca-cert\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.417852 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.417827    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-envoy\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.418053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.418035    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-podinfo\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.425015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.424987    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m4gl\" (UniqueName: \"kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-kube-api-access-2m4gl\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.427345 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.427325    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-token\") pod \"e2e-noten-8e4b08-openshift-default-6cc7859754-hv996\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") " pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.514223 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.514132    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:03.790747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.790714    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"]
Sep 10 21:27:03.791520 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:27:03.791497    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod780c8e3a_0a20_42e3_ae04_33dca49f69b1.slice/crio-5a48b3edde124d78101dc1060210b65a484a7ba987bcbde8ee7c12e3b16729f9 WatchSource:0}: Error finding container 5a48b3edde124d78101dc1060210b65a484a7ba987bcbde8ee7c12e3b16729f9: Status 404 returned error can't find the container with id 5a48b3edde124d78101dc1060210b65a484a7ba987bcbde8ee7c12e3b16729f9
Sep 10 21:27:03.793621 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.793589    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:27:03.793674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.793665    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:27:03.793709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.793695    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:27:03.918269 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.918228    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996" event={"ID":"780c8e3a-0a20-42e3-ae04-33dca49f69b1","Type":"ContainerStarted","Data":"be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360"}
Sep 10 21:27:03.918434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.918290    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996" event={"ID":"780c8e3a-0a20-42e3-ae04-33dca49f69b1","Type":"ContainerStarted","Data":"5a48b3edde124d78101dc1060210b65a484a7ba987bcbde8ee7c12e3b16729f9"}
Sep 10 21:27:03.939029 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:03.938936    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996" podStartSLOduration=0.938922146 podStartE2EDuration="938.922146ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:27:03.938606829 +0000 UTC m=+999.872273683" watchObservedRunningTime="2026-09-10 21:27:03.938922146 +0000 UTC m=+999.872588955"
Sep 10 21:27:04.514276 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:04.514238    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:04.519387 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:04.519360    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:04.923820 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:04.923771    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:04.924814 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:04.924771    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:07.214235 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.214199    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:27:07.697572 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.697543    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:27:07.759548 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.759460    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6f4h\" (UniqueName: \"kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-kube-api-access-n6f4h\") pod \"581e9ccd-1552-44ce-96c9-5e148f54caa0\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") "
Sep 10 21:27:07.759720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.759546    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-envoy\") pod \"581e9ccd-1552-44ce-96c9-5e148f54caa0\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") "
Sep 10 21:27:07.759720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.759579    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-podinfo\") pod \"581e9ccd-1552-44ce-96c9-5e148f54caa0\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") "
Sep 10 21:27:07.759720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.759647    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-token\") pod \"581e9ccd-1552-44ce-96c9-5e148f54caa0\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") "
Sep 10 21:27:07.759720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.759688    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581e9ccd-1552-44ce-96c9-5e148f54caa0-istiod-ca-cert\") pod \"581e9ccd-1552-44ce-96c9-5e148f54caa0\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") "
Sep 10 21:27:07.759720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.759711    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-data\") pod \"581e9ccd-1552-44ce-96c9-5e148f54caa0\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") "
Sep 10 21:27:07.760136 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.760088    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-data" (OuterVolumeSpecName: "istio-data") pod "581e9ccd-1552-44ce-96c9-5e148f54caa0" (UID: "581e9ccd-1552-44ce-96c9-5e148f54caa0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:07.760251 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.760187    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-certs\") pod \"581e9ccd-1552-44ce-96c9-5e148f54caa0\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") "
Sep 10 21:27:07.760337 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.760319    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-socket\") pod \"581e9ccd-1552-44ce-96c9-5e148f54caa0\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") "
Sep 10 21:27:07.760469 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.760454    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-credential-socket\") pod \"581e9ccd-1552-44ce-96c9-5e148f54caa0\" (UID: \"581e9ccd-1552-44ce-96c9-5e148f54caa0\") "
Sep 10 21:27:07.760585 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.760241    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/581e9ccd-1552-44ce-96c9-5e148f54caa0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "581e9ccd-1552-44ce-96c9-5e148f54caa0" (UID: "581e9ccd-1552-44ce-96c9-5e148f54caa0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:27:07.760906 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.760845    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:07.760906 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.760387    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "581e9ccd-1552-44ce-96c9-5e148f54caa0" (UID: "581e9ccd-1552-44ce-96c9-5e148f54caa0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:07.760906 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.760611    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "581e9ccd-1552-44ce-96c9-5e148f54caa0" (UID: "581e9ccd-1552-44ce-96c9-5e148f54caa0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:07.760906 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.760845    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "581e9ccd-1552-44ce-96c9-5e148f54caa0" (UID: "581e9ccd-1552-44ce-96c9-5e148f54caa0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:07.762408 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.762372    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "581e9ccd-1552-44ce-96c9-5e148f54caa0" (UID: "581e9ccd-1552-44ce-96c9-5e148f54caa0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:27:07.762408 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.762375    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-kube-api-access-n6f4h" (OuterVolumeSpecName: "kube-api-access-n6f4h") pod "581e9ccd-1552-44ce-96c9-5e148f54caa0" (UID: "581e9ccd-1552-44ce-96c9-5e148f54caa0"). InnerVolumeSpecName "kube-api-access-n6f4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:27:07.762946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.762923    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "581e9ccd-1552-44ce-96c9-5e148f54caa0" (UID: "581e9ccd-1552-44ce-96c9-5e148f54caa0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:07.763042 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.763014    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-token" (OuterVolumeSpecName: "istio-token") pod "581e9ccd-1552-44ce-96c9-5e148f54caa0" (UID: "581e9ccd-1552-44ce-96c9-5e148f54caa0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:27:07.861765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.861726    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:07.861765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.861760    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:07.861765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.861769    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:07.862019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.861802    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6f4h\" (UniqueName: \"kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-kube-api-access-n6f4h\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:07.862019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.861816    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:07.862019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.861827    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:07.862019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.861834    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581e9ccd-1552-44ce-96c9-5e148f54caa0-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:07.862019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.861842    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581e9ccd-1552-44ce-96c9-5e148f54caa0-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:07.942700 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.942670    2573 generic.go:356] "Generic (PLEG): container finished" podID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerID="a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e" exitCode=0
Sep 10 21:27:07.942872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.942726    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" event={"ID":"581e9ccd-1552-44ce-96c9-5e148f54caa0","Type":"ContainerDied","Data":"a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e"}
Sep 10 21:27:07.942872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.942735    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"
Sep 10 21:27:07.942872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.942804    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk" event={"ID":"581e9ccd-1552-44ce-96c9-5e148f54caa0","Type":"ContainerDied","Data":"c3aad615964921058fc93b8b8de38147414199fb1242dab8ca663dd959e085ce"}
Sep 10 21:27:07.942872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.942827    2573 scope.go:117] "RemoveContainer" containerID="a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e"
Sep 10 21:27:07.953208 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.953184    2573 scope.go:117] "RemoveContainer" containerID="9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80"
Sep 10 21:27:07.961867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.961845    2573 scope.go:117] "RemoveContainer" containerID="a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e"
Sep 10 21:27:07.962138 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:27:07.962119    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e\": container with ID starting with a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e not found: ID does not exist" containerID="a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e"
Sep 10 21:27:07.962199 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.962146    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e"} err="failed to get container status \"a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e\": rpc error: code = NotFound desc = could not find container \"a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e\": container with ID starting with a820c02ae350ff34602e25b406cb3039c9aeb760fffe33c79e1d95b8620fa87e not found: ID does not exist"
Sep 10 21:27:07.962199 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.962165    2573 scope.go:117] "RemoveContainer" containerID="9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80"
Sep 10 21:27:07.962437 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:27:07.962420    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80\": container with ID starting with 9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80 not found: ID does not exist" containerID="9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80"
Sep 10 21:27:07.962483 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.962444    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80"} err="failed to get container status \"9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80\": rpc error: code = NotFound desc = could not find container \"9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80\": container with ID starting with 9616b6009683aff13b2b68922defc997ae4c7b2306e077610919c19ae63a6b80 not found: ID does not exist"
Sep 10 21:27:07.969749 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.969727    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"]
Sep 10 21:27:07.987074 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:07.987051    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-fbe4cf-openshift-default-646d44b84d-tvvrk"]
Sep 10 21:27:08.678788 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:08.678753    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" path="/var/lib/kubelet/pods/581e9ccd-1552-44ce-96c9-5e148f54caa0/volumes"
Sep 10 21:27:17.713006 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.712967    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"]
Sep 10 21:27:17.713574 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.713324    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy"
Sep 10 21:27:17.713574 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.713335    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy"
Sep 10 21:27:17.713574 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.713348    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy"
Sep 10 21:27:17.713574 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.713355    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy"
Sep 10 21:27:17.713574 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.713429    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy"
Sep 10 21:27:17.713574 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.713437    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="581e9ccd-1552-44ce-96c9-5e148f54caa0" containerName="istio-proxy"
Sep 10 21:27:17.716840 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.716820    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.719745 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.719715    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-8e4b08\"/\"kube-root-ca.crt\""
Sep 10 21:27:17.719893 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.719748    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-8e4b08\"/\"openshift-service-ca.crt\""
Sep 10 21:27:17.719893 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.719716    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-8e4b08\"/\"noten-model-8e4b08-kserve-self-signed-certs\""
Sep 10 21:27:17.719893 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.719716    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-8e4b08\"/\"default-dockercfg-tdn4d\""
Sep 10 21:27:17.727581 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.727559    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"]
Sep 10 21:27:17.851617 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.851568    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-home\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.851866 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.851660    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-tmp-dir\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.851866 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.851700    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btq9h\" (UniqueName: \"kubernetes.io/projected/84820544-974b-46ae-bc3b-4cae7fd97820-kube-api-access-btq9h\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.851866 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.851766    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-model-cache\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.851866 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.851841    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-dshm\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.851866 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.851860    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84820544-974b-46ae-bc3b-4cae7fd97820-tls-certs\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.952978 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.952936    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-home\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.953159 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.953024    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-tmp-dir\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.953159 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.953067    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btq9h\" (UniqueName: \"kubernetes.io/projected/84820544-974b-46ae-bc3b-4cae7fd97820-kube-api-access-btq9h\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.953159 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.953105    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-model-cache\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.953159 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.953144    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-dshm\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.953353 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.953167    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84820544-974b-46ae-bc3b-4cae7fd97820-tls-certs\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.953451 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.953428    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-home\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.953514 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.953438    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-tmp-dir\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.953514 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.953500    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-model-cache\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.955427 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.955396    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-dshm\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.955679 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.955659    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84820544-974b-46ae-bc3b-4cae7fd97820-tls-certs\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:17.963499 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:17.963444    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btq9h\" (UniqueName: \"kubernetes.io/projected/84820544-974b-46ae-bc3b-4cae7fd97820-kube-api-access-btq9h\") pod \"noten-model-8e4b08-kserve-9b658b6ff-45wp8\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") " pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:18.027119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:18.027074    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:18.297464 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:18.297436    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"]
Sep 10 21:27:18.299420 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:27:18.299388    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84820544_974b_46ae_bc3b_4cae7fd97820.slice/crio-13c148cad4735cc133ee43ca2319ddcf1ac3c97c01e9af11818fa5438b5949e2 WatchSource:0}: Error finding container 13c148cad4735cc133ee43ca2319ddcf1ac3c97c01e9af11818fa5438b5949e2: Status 404 returned error can't find the container with id 13c148cad4735cc133ee43ca2319ddcf1ac3c97c01e9af11818fa5438b5949e2
Sep 10 21:27:19.006361 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:19.006317    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8" event={"ID":"84820544-974b-46ae-bc3b-4cae7fd97820","Type":"ContainerStarted","Data":"6ad6e1bd4d6530fff7781eb3646bf155f1e9652397ef09fab8cb4c79a722be3d"}
Sep 10 21:27:19.006729 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:19.006371    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8" event={"ID":"84820544-974b-46ae-bc3b-4cae7fd97820","Type":"ContainerStarted","Data":"13c148cad4735cc133ee43ca2319ddcf1ac3c97c01e9af11818fa5438b5949e2"}
Sep 10 21:27:19.024020 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:19.023973    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8" podStartSLOduration=2.023957442 podStartE2EDuration="2.023957442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:27:19.022843389 +0000 UTC m=+1014.956510205" watchObservedRunningTime="2026-09-10 21:27:19.023957442 +0000 UTC m=+1014.957624269"
Sep 10 21:27:20.421002 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:20.420965    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:27:26.965463 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:26.965428    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"]
Sep 10 21:27:27.084699 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:27.084652    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"]
Sep 10 21:27:27.085011 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:27.084962    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996" podUID="780c8e3a-0a20-42e3-ae04-33dca49f69b1" containerName="istio-proxy" containerID="cri-o://be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360" gracePeriod=30
Sep 10 21:27:28.027751 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:28.027708    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:31.324061 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:31.324017    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:27:32.322617 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.322592    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:32.389473 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389384    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-credential-socket\") pod \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") "
Sep 10 21:27:32.389473 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389437    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-podinfo\") pod \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") "
Sep 10 21:27:32.389473 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389460    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istiod-ca-cert\") pod \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") "
Sep 10 21:27:32.390080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389484    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-envoy\") pod \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") "
Sep 10 21:27:32.390080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389514    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m4gl\" (UniqueName: \"kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-kube-api-access-2m4gl\") pod \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") "
Sep 10 21:27:32.390080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389539    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-token\") pod \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") "
Sep 10 21:27:32.390080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389573    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-socket\") pod \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") "
Sep 10 21:27:32.390080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389603    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-certs\") pod \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") "
Sep 10 21:27:32.390080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389640    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-data\") pod \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\" (UID: \"780c8e3a-0a20-42e3-ae04-33dca49f69b1\") "
Sep 10 21:27:32.390080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389740    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "780c8e3a-0a20-42e3-ae04-33dca49f69b1" (UID: "780c8e3a-0a20-42e3-ae04-33dca49f69b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:32.390080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.389989    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-data" (OuterVolumeSpecName: "istio-data") pod "780c8e3a-0a20-42e3-ae04-33dca49f69b1" (UID: "780c8e3a-0a20-42e3-ae04-33dca49f69b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:32.390080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.390011    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:32.390080 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.390017    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "780c8e3a-0a20-42e3-ae04-33dca49f69b1" (UID: "780c8e3a-0a20-42e3-ae04-33dca49f69b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:27:32.390626 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.390215    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "780c8e3a-0a20-42e3-ae04-33dca49f69b1" (UID: "780c8e3a-0a20-42e3-ae04-33dca49f69b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:32.390626 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.390229    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "780c8e3a-0a20-42e3-ae04-33dca49f69b1" (UID: "780c8e3a-0a20-42e3-ae04-33dca49f69b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:32.391745 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.391715    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "780c8e3a-0a20-42e3-ae04-33dca49f69b1" (UID: "780c8e3a-0a20-42e3-ae04-33dca49f69b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:27:32.392001 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.391970    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "780c8e3a-0a20-42e3-ae04-33dca49f69b1" (UID: "780c8e3a-0a20-42e3-ae04-33dca49f69b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:32.392001 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.391992    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-token" (OuterVolumeSpecName: "istio-token") pod "780c8e3a-0a20-42e3-ae04-33dca49f69b1" (UID: "780c8e3a-0a20-42e3-ae04-33dca49f69b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:27:32.392130 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.392018    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-kube-api-access-2m4gl" (OuterVolumeSpecName: "kube-api-access-2m4gl") pod "780c8e3a-0a20-42e3-ae04-33dca49f69b1" (UID: "780c8e3a-0a20-42e3-ae04-33dca49f69b1"). InnerVolumeSpecName "kube-api-access-2m4gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:27:32.490727 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.490691    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:32.490727 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.490722    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m4gl\" (UniqueName: \"kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-kube-api-access-2m4gl\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:32.490727 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.490732    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:32.490985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.490743    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:32.490985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.490751    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:32.490985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.490760    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:32.490985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.490768    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:32.490985 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:32.490812    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780c8e3a-0a20-42e3-ae04-33dca49f69b1-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:33.085400 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:33.085365    2573 generic.go:356] "Generic (PLEG): container finished" podID="780c8e3a-0a20-42e3-ae04-33dca49f69b1" containerID="be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360" exitCode=0
Sep 10 21:27:33.085600 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:33.085406    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"
Sep 10 21:27:33.085600 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:33.085451    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996" event={"ID":"780c8e3a-0a20-42e3-ae04-33dca49f69b1","Type":"ContainerDied","Data":"be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360"}
Sep 10 21:27:33.085600 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:33.085514    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996" event={"ID":"780c8e3a-0a20-42e3-ae04-33dca49f69b1","Type":"ContainerDied","Data":"5a48b3edde124d78101dc1060210b65a484a7ba987bcbde8ee7c12e3b16729f9"}
Sep 10 21:27:33.085600 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:33.085543    2573 scope.go:117] "RemoveContainer" containerID="be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360"
Sep 10 21:27:33.094813 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:33.094761    2573 scope.go:117] "RemoveContainer" containerID="be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360"
Sep 10 21:27:33.095054 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:27:33.095034    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360\": container with ID starting with be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360 not found: ID does not exist" containerID="be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360"
Sep 10 21:27:33.095103 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:33.095066    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360"} err="failed to get container status \"be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360\": rpc error: code = NotFound desc = could not find container \"be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360\": container with ID starting with be1dd95038ef56a360e6cbfa14e4ec380ccb73ab99ee8f19f21eaa80bd322360 not found: ID does not exist"
Sep 10 21:27:33.105802 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:33.105518    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"]
Sep 10 21:27:33.111003 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:33.110971    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-8e4b08-openshift-default-6cc7859754-hv996"]
Sep 10 21:27:34.679118 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:34.679077    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="780c8e3a-0a20-42e3-ae04-33dca49f69b1" path="/var/lib/kubelet/pods/780c8e3a-0a20-42e3-ae04-33dca49f69b1/volumes"
Sep 10 21:27:40.878954 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.878920    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"]
Sep 10 21:27:40.879329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.879280    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="780c8e3a-0a20-42e3-ae04-33dca49f69b1" containerName="istio-proxy"
Sep 10 21:27:40.879329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.879290    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="780c8e3a-0a20-42e3-ae04-33dca49f69b1" containerName="istio-proxy"
Sep 10 21:27:40.879416 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.879353    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="780c8e3a-0a20-42e3-ae04-33dca49f69b1" containerName="istio-proxy"
Sep 10 21:27:40.882409 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.882391    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:40.884714 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.884683    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-76320447-openshift-default-dockercfg-tjh47\""
Sep 10 21:27:40.893986 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.893961    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"]
Sep 10 21:27:40.967226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.967186    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-certs\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:40.967226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.967232    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-data\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:40.967434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.967251    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istiod-ca-cert\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:40.967434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.967268    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-podinfo\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:40.967434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.967283    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42qn8\" (UniqueName: \"kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-kube-api-access-42qn8\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:40.967434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.967327    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-credential-socket\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:40.967434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.967370    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-socket\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:40.967614 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.967484    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-envoy\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:40.967614 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:40.967553    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-token\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.068960 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.068915    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-credential-socket\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.068960 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.068963    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-socket\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069163 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.068988    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-envoy\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069163 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.069106    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-token\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069235 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.069191    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-certs\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069269 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.069229    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-data\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069269 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.069262    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istiod-ca-cert\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069367 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.069290    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-podinfo\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069367 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.069315    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42qn8\" (UniqueName: \"kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-kube-api-access-42qn8\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069367 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.069350    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-credential-socket\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069554 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.069534    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-certs\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069620 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.069607    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-data\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.069736 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.069718    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-socket\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.070050 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.070027    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istiod-ca-cert\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.071262 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.071239    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-envoy\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.071751 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.071717    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-podinfo\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.087937 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.087908    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-token\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.088537 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.088516    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42qn8\" (UniqueName: \"kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-kube-api-access-42qn8\") pod \"e2e-mt-76320447-openshift-default-5bb897b888-ff464\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") " pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.193728 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.193626    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:41.469485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.469459    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"]
Sep 10 21:27:41.470140 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:27:41.470113    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e5e4128_9038_4d8e_9af9_24bc5920abd7.slice/crio-6303f1296e127533a1fa473d323178700f2ef2a0b30320e7bf6d0aea2fb1c5e4 WatchSource:0}: Error finding container 6303f1296e127533a1fa473d323178700f2ef2a0b30320e7bf6d0aea2fb1c5e4: Status 404 returned error can't find the container with id 6303f1296e127533a1fa473d323178700f2ef2a0b30320e7bf6d0aea2fb1c5e4
Sep 10 21:27:41.472321 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.472291    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:27:41.472399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.472353    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:27:41.472399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.472381    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:27:41.985297 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:41.985233    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8" podUID="84820544-974b-46ae-bc3b-4cae7fd97820" containerName="main" containerID="cri-o://6ad6e1bd4d6530fff7781eb3646bf155f1e9652397ef09fab8cb4c79a722be3d" gracePeriod=45
Sep 10 21:27:42.142499 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.142468    2573 generic.go:356] "Generic (PLEG): container finished" podID="84820544-974b-46ae-bc3b-4cae7fd97820" containerID="6ad6e1bd4d6530fff7781eb3646bf155f1e9652397ef09fab8cb4c79a722be3d" exitCode=0
Sep 10 21:27:42.142674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.142547    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8" event={"ID":"84820544-974b-46ae-bc3b-4cae7fd97820","Type":"ContainerDied","Data":"6ad6e1bd4d6530fff7781eb3646bf155f1e9652397ef09fab8cb4c79a722be3d"}
Sep 10 21:27:42.144373 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.144352    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" event={"ID":"4e5e4128-9038-4d8e-9af9-24bc5920abd7","Type":"ContainerStarted","Data":"58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967"}
Sep 10 21:27:42.144461 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.144391    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" event={"ID":"4e5e4128-9038-4d8e-9af9-24bc5920abd7","Type":"ContainerStarted","Data":"6303f1296e127533a1fa473d323178700f2ef2a0b30320e7bf6d0aea2fb1c5e4"}
Sep 10 21:27:42.172676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.172626    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" podStartSLOduration=2.172607616 podStartE2EDuration="2.172607616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:27:42.170868885 +0000 UTC m=+1038.104535700" watchObservedRunningTime="2026-09-10 21:27:42.172607616 +0000 UTC m=+1038.106274431"
Sep 10 21:27:42.194632 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.194595    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:42.199821 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.199792    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:42.242990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.242925    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:42.281221 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281188    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-home\") pod \"84820544-974b-46ae-bc3b-4cae7fd97820\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") "
Sep 10 21:27:42.281399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281253    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-tmp-dir\") pod \"84820544-974b-46ae-bc3b-4cae7fd97820\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") "
Sep 10 21:27:42.281399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281282    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-model-cache\") pod \"84820544-974b-46ae-bc3b-4cae7fd97820\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") "
Sep 10 21:27:42.281399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281310    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84820544-974b-46ae-bc3b-4cae7fd97820-tls-certs\") pod \"84820544-974b-46ae-bc3b-4cae7fd97820\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") "
Sep 10 21:27:42.281399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281344    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-dshm\") pod \"84820544-974b-46ae-bc3b-4cae7fd97820\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") "
Sep 10 21:27:42.281399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281395    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btq9h\" (UniqueName: \"kubernetes.io/projected/84820544-974b-46ae-bc3b-4cae7fd97820-kube-api-access-btq9h\") pod \"84820544-974b-46ae-bc3b-4cae7fd97820\" (UID: \"84820544-974b-46ae-bc3b-4cae7fd97820\") "
Sep 10 21:27:42.281653 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281447    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-home" (OuterVolumeSpecName: "home") pod "84820544-974b-46ae-bc3b-4cae7fd97820" (UID: "84820544-974b-46ae-bc3b-4cae7fd97820"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:42.281653 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281486    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84820544-974b-46ae-bc3b-4cae7fd97820" (UID: "84820544-974b-46ae-bc3b-4cae7fd97820"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:42.281653 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281572    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-model-cache" (OuterVolumeSpecName: "model-cache") pod "84820544-974b-46ae-bc3b-4cae7fd97820" (UID: "84820544-974b-46ae-bc3b-4cae7fd97820"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:42.281813 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281797    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:42.281873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281822    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:42.281873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.281835    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:42.283669 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.283640    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-dshm" (OuterVolumeSpecName: "dshm") pod "84820544-974b-46ae-bc3b-4cae7fd97820" (UID: "84820544-974b-46ae-bc3b-4cae7fd97820"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:27:42.283669 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.283654    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84820544-974b-46ae-bc3b-4cae7fd97820-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84820544-974b-46ae-bc3b-4cae7fd97820" (UID: "84820544-974b-46ae-bc3b-4cae7fd97820"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:27:42.283875 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.283758    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84820544-974b-46ae-bc3b-4cae7fd97820-kube-api-access-btq9h" (OuterVolumeSpecName: "kube-api-access-btq9h") pod "84820544-974b-46ae-bc3b-4cae7fd97820" (UID: "84820544-974b-46ae-bc3b-4cae7fd97820"). InnerVolumeSpecName "kube-api-access-btq9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:27:42.383256 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.383222    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btq9h\" (UniqueName: \"kubernetes.io/projected/84820544-974b-46ae-bc3b-4cae7fd97820-kube-api-access-btq9h\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:42.383256 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.383252    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84820544-974b-46ae-bc3b-4cae7fd97820-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:42.383256 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:42.383263    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84820544-974b-46ae-bc3b-4cae7fd97820-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:27:43.151045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:43.151014    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"
Sep 10 21:27:43.151515 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:43.151062    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8" event={"ID":"84820544-974b-46ae-bc3b-4cae7fd97820","Type":"ContainerDied","Data":"13c148cad4735cc133ee43ca2319ddcf1ac3c97c01e9af11818fa5438b5949e2"}
Sep 10 21:27:43.151515 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:43.151114    2573 scope.go:117] "RemoveContainer" containerID="6ad6e1bd4d6530fff7781eb3646bf155f1e9652397ef09fab8cb4c79a722be3d"
Sep 10 21:27:43.151631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:43.151568    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:43.152747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:43.152719    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:27:43.169973 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:43.169946    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"]
Sep 10 21:27:43.174184 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:43.174161    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-8e4b08/noten-model-8e4b08-kserve-9b658b6ff-45wp8"]
Sep 10 21:27:44.678653 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:44.678621    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84820544-974b-46ae-bc3b-4cae7fd97820" path="/var/lib/kubelet/pods/84820544-974b-46ae-bc3b-4cae7fd97820/volumes"
Sep 10 21:27:45.819719 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:45.819683    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:27:50.133972 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:50.133927    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"]
Sep 10 21:27:58.509854 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:27:58.509812    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:28:03.269358 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:03.269324    2573 generic.go:356] "Generic (PLEG): container finished" podID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerID="58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967" exitCode=0
Sep 10 21:28:03.269898 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:03.269402    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" event={"ID":"4e5e4128-9038-4d8e-9af9-24bc5920abd7","Type":"ContainerDied","Data":"58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967"}
Sep 10 21:28:03.269974 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:03.269932    2573 scope.go:117] "RemoveContainer" containerID="58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967"
Sep 10 21:28:03.271052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:03.271023    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:28:03.271156 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:03.271094    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:28:03.271156 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:03.271125    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:28:04.277417 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:04.277376    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" event={"ID":"4e5e4128-9038-4d8e-9af9-24bc5920abd7","Type":"ContainerStarted","Data":"1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5"}
Sep 10 21:28:05.153240 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.153193    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8" podUID="6780bcb1-34ff-4dd3-81d1-1888b9d1779b" containerName="main" containerID="cri-o://e92d98366ffdf6829288cfe8280c3069c6d2c47990df0be2ad98a3d2b96b0bf0" gracePeriod=45
Sep 10 21:28:05.194594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.194545    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:28:05.195033 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.195003    2573 patch_prober.go:28] interesting pod/e2e-mt-76320447-openshift-default-5bb897b888-ff464 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 10 21:28:05.195148 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.195070    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 10 21:28:05.299501 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.299445    2573 generic.go:356] "Generic (PLEG): container finished" podID="6780bcb1-34ff-4dd3-81d1-1888b9d1779b" containerID="e92d98366ffdf6829288cfe8280c3069c6d2c47990df0be2ad98a3d2b96b0bf0" exitCode=0
Sep 10 21:28:05.300516 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.299522    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8" event={"ID":"6780bcb1-34ff-4dd3-81d1-1888b9d1779b","Type":"ContainerDied","Data":"e92d98366ffdf6829288cfe8280c3069c6d2c47990df0be2ad98a3d2b96b0bf0"}
Sep 10 21:28:05.447510 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.447484    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:28:05.519880 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.519845    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-model-cache\") pod \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") "
Sep 10 21:28:05.520059 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.519905    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-home\") pod \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") "
Sep 10 21:28:05.520059 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.519961    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmxh8\" (UniqueName: \"kubernetes.io/projected/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-kube-api-access-qmxh8\") pod \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") "
Sep 10 21:28:05.520059 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.519999    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-dshm\") pod \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") "
Sep 10 21:28:05.520059 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.520030    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tls-certs\") pod \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") "
Sep 10 21:28:05.520290 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.520084    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tmp-dir\") pod \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\" (UID: \"6780bcb1-34ff-4dd3-81d1-1888b9d1779b\") "
Sep 10 21:28:05.520290 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.520152    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-model-cache" (OuterVolumeSpecName: "model-cache") pod "6780bcb1-34ff-4dd3-81d1-1888b9d1779b" (UID: "6780bcb1-34ff-4dd3-81d1-1888b9d1779b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:05.520290 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.520199    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-home" (OuterVolumeSpecName: "home") pod "6780bcb1-34ff-4dd3-81d1-1888b9d1779b" (UID: "6780bcb1-34ff-4dd3-81d1-1888b9d1779b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:05.520460 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.520349    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6780bcb1-34ff-4dd3-81d1-1888b9d1779b" (UID: "6780bcb1-34ff-4dd3-81d1-1888b9d1779b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:05.520460 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.520417    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:05.520460 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.520438    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:05.520460 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.520456    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:05.522317 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.522289    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-dshm" (OuterVolumeSpecName: "dshm") pod "6780bcb1-34ff-4dd3-81d1-1888b9d1779b" (UID: "6780bcb1-34ff-4dd3-81d1-1888b9d1779b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:05.522440 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.522328    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6780bcb1-34ff-4dd3-81d1-1888b9d1779b" (UID: "6780bcb1-34ff-4dd3-81d1-1888b9d1779b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:05.522440 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.522333    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-kube-api-access-qmxh8" (OuterVolumeSpecName: "kube-api-access-qmxh8") pod "6780bcb1-34ff-4dd3-81d1-1888b9d1779b" (UID: "6780bcb1-34ff-4dd3-81d1-1888b9d1779b"). InnerVolumeSpecName "kube-api-access-qmxh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:05.621456 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.621410    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmxh8\" (UniqueName: \"kubernetes.io/projected/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-kube-api-access-qmxh8\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:05.621456 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.621446    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:05.621456 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:05.621458    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6780bcb1-34ff-4dd3-81d1-1888b9d1779b-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:06.194965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:06.194921    2573 patch_prober.go:28] interesting pod/e2e-mt-76320447-openshift-default-5bb897b888-ff464 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 10 21:28:06.195158 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:06.194991    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 10 21:28:06.306897 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:06.306867    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"
Sep 10 21:28:06.307331 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:06.306867    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8" event={"ID":"6780bcb1-34ff-4dd3-81d1-1888b9d1779b","Type":"ContainerDied","Data":"5e8dd7867ef69550b44680ce309a919fc2fe8292b9070ae1322a66905e8a409c"}
Sep 10 21:28:06.307331 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:06.306994    2573 scope.go:117] "RemoveContainer" containerID="e92d98366ffdf6829288cfe8280c3069c6d2c47990df0be2ad98a3d2b96b0bf0"
Sep 10 21:28:06.328701 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:06.328669    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"]
Sep 10 21:28:06.332933 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:06.332901    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a6441b/test-model-a6441b-kserve-5569dd99f7-j6qp8"]
Sep 10 21:28:06.682698 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:06.682654    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6780bcb1-34ff-4dd3-81d1-1888b9d1779b" path="/var/lib/kubelet/pods/6780bcb1-34ff-4dd3-81d1-1888b9d1779b/volumes"
Sep 10 21:28:07.195248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:07.194803    2573 patch_prober.go:28] interesting pod/e2e-mt-76320447-openshift-default-5bb897b888-ff464 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 10 21:28:07.195248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:07.194866    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 10 21:28:08.194714 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:08.194244    2573 patch_prober.go:28] interesting pod/e2e-mt-76320447-openshift-default-5bb897b888-ff464 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 10 21:28:08.194714 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:08.194306    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 10 21:28:08.828729 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:08.828695    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:28:09.194856 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:09.194730    2573 patch_prober.go:28] interesting pod/e2e-mt-76320447-openshift-default-5bb897b888-ff464 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 10 21:28:09.195424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:09.195392    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 10 21:28:10.197921 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:10.197890    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:28:10.198295 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:10.198177    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:28:10.198969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:10.198950    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:28:19.456547 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.456470    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"]
Sep 10 21:28:19.456999 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.456686    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l" podUID="5dd07a8c-df59-480d-982c-f69adc174e72" containerName="maas-api" containerID="cri-o://e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af" gracePeriod=30
Sep 10 21:28:19.695276 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.695251    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:28:19.750475 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.750406    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd07a8c-df59-480d-982c-f69adc174e72-openshift-service-ca\") pod \"5dd07a8c-df59-480d-982c-f69adc174e72\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") "
Sep 10 21:28:19.750475 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.750446    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls\") pod \"5dd07a8c-df59-480d-982c-f69adc174e72\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") "
Sep 10 21:28:19.750651 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.750500    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-metrics-certs\") pod \"5dd07a8c-df59-480d-982c-f69adc174e72\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") "
Sep 10 21:28:19.750651 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.750535    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4pz7\" (UniqueName: \"kubernetes.io/projected/5dd07a8c-df59-480d-982c-f69adc174e72-kube-api-access-w4pz7\") pod \"5dd07a8c-df59-480d-982c-f69adc174e72\" (UID: \"5dd07a8c-df59-480d-982c-f69adc174e72\") "
Sep 10 21:28:19.750820 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.750795    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd07a8c-df59-480d-982c-f69adc174e72-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5dd07a8c-df59-480d-982c-f69adc174e72" (UID: "5dd07a8c-df59-480d-982c-f69adc174e72"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:19.750950 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.750930    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd07a8c-df59-480d-982c-f69adc174e72-openshift-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:19.752720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.752689    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dd07a8c-df59-480d-982c-f69adc174e72-kube-api-access-w4pz7" (OuterVolumeSpecName: "kube-api-access-w4pz7") pod "5dd07a8c-df59-480d-982c-f69adc174e72" (UID: "5dd07a8c-df59-480d-982c-f69adc174e72"). InnerVolumeSpecName "kube-api-access-w4pz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:19.752823 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.752708    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5dd07a8c-df59-480d-982c-f69adc174e72" (UID: "5dd07a8c-df59-480d-982c-f69adc174e72"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:19.752823 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.752720    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5dd07a8c-df59-480d-982c-f69adc174e72" (UID: "5dd07a8c-df59-480d-982c-f69adc174e72"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:19.852057 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.852023    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-metrics-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:19.852057 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.852052    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4pz7\" (UniqueName: \"kubernetes.io/projected/5dd07a8c-df59-480d-982c-f69adc174e72-kube-api-access-w4pz7\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:19.852057 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:19.852062    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd07a8c-df59-480d-982c-f69adc174e72-maas-api-tls\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:20.389835 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:20.389797    2573 generic.go:356] "Generic (PLEG): container finished" podID="5dd07a8c-df59-480d-982c-f69adc174e72" containerID="e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af" exitCode=0
Sep 10 21:28:20.390010 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:20.389838    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"
Sep 10 21:28:20.390010 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:20.389868    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l" event={"ID":"5dd07a8c-df59-480d-982c-f69adc174e72","Type":"ContainerDied","Data":"e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af"}
Sep 10 21:28:20.390010 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:20.389928    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l" event={"ID":"5dd07a8c-df59-480d-982c-f69adc174e72","Type":"ContainerDied","Data":"e4db5adbc966abc2bd4cc232840dceeb86b4de0b45c4a0b6e33919b700355c14"}
Sep 10 21:28:20.390010 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:20.389959    2573 scope.go:117] "RemoveContainer" containerID="e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af"
Sep 10 21:28:20.400248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:20.400215    2573 scope.go:117] "RemoveContainer" containerID="e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af"
Sep 10 21:28:20.405302 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:28:20.405267    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af\": container with ID starting with e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af not found: ID does not exist" containerID="e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af"
Sep 10 21:28:20.405457 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:20.405310    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af"} err="failed to get container status \"e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af\": rpc error: code = NotFound desc = could not find container \"e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af\": container with ID starting with e75cbe57a26aa445ffb4ed2e2ed3f52f253c8bc8e0955a31ab85f47480f083af not found: ID does not exist"
Sep 10 21:28:20.415071 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:20.415043    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"]
Sep 10 21:28:20.417154 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:20.417134    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l"]
Sep 10 21:28:20.679896 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:20.679812    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dd07a8c-df59-480d-982c-f69adc174e72" path="/var/lib/kubelet/pods/5dd07a8c-df59-480d-982c-f69adc174e72/volumes"
Sep 10 21:28:22.415661 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:22.415627    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:28:22.652413 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:22.652358    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6441b-564c74d4f7-88j5l" podUID="5dd07a8c-df59-480d-982c-f69adc174e72" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.47:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 10 21:28:26.436372 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:26.436337    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"]
Sep 10 21:28:26.436798 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:26.436585    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy" containerID="cri-o://a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413" gracePeriod=30
Sep 10 21:28:30.731174 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:30.731136    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:28:31.686943 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.686918    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:28:31.763172 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763087    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-istio-token\") pod \"c547695c-7b06-419c-9de4-dec615374497\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") "
Sep 10 21:28:31.763172 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763121    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsmzn\" (UniqueName: \"kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-kube-api-access-qsmzn\") pod \"c547695c-7b06-419c-9de4-dec615374497\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") "
Sep 10 21:28:31.763172 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763151    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-credential-socket\") pod \"c547695c-7b06-419c-9de4-dec615374497\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") "
Sep 10 21:28:31.763674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763175    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-socket\") pod \"c547695c-7b06-419c-9de4-dec615374497\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") "
Sep 10 21:28:31.763674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763218    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c547695c-7b06-419c-9de4-dec615374497-istio-podinfo\") pod \"c547695c-7b06-419c-9de4-dec615374497\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") "
Sep 10 21:28:31.763674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763240    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-data\") pod \"c547695c-7b06-419c-9de4-dec615374497\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") "
Sep 10 21:28:31.763674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763267    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-envoy\") pod \"c547695c-7b06-419c-9de4-dec615374497\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") "
Sep 10 21:28:31.763674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763308    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c547695c-7b06-419c-9de4-dec615374497-istiod-ca-cert\") pod \"c547695c-7b06-419c-9de4-dec615374497\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") "
Sep 10 21:28:31.763674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763356    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-certs\") pod \"c547695c-7b06-419c-9de4-dec615374497\" (UID: \"c547695c-7b06-419c-9de4-dec615374497\") "
Sep 10 21:28:31.763674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763390    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c547695c-7b06-419c-9de4-dec615374497" (UID: "c547695c-7b06-419c-9de4-dec615374497"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:31.763674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763616    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-data" (OuterVolumeSpecName: "istio-data") pod "c547695c-7b06-419c-9de4-dec615374497" (UID: "c547695c-7b06-419c-9de4-dec615374497"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:31.763674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763638    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:31.764219 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763737    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c547695c-7b06-419c-9de4-dec615374497-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c547695c-7b06-419c-9de4-dec615374497" (UID: "c547695c-7b06-419c-9de4-dec615374497"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:31.764219 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763802    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c547695c-7b06-419c-9de4-dec615374497" (UID: "c547695c-7b06-419c-9de4-dec615374497"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:31.764219 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.763881    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c547695c-7b06-419c-9de4-dec615374497" (UID: "c547695c-7b06-419c-9de4-dec615374497"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:31.765483 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.765458    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-kube-api-access-qsmzn" (OuterVolumeSpecName: "kube-api-access-qsmzn") pod "c547695c-7b06-419c-9de4-dec615374497" (UID: "c547695c-7b06-419c-9de4-dec615374497"). InnerVolumeSpecName "kube-api-access-qsmzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:31.765594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.765554    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c547695c-7b06-419c-9de4-dec615374497" (UID: "c547695c-7b06-419c-9de4-dec615374497"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:31.765882 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.765849    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c547695c-7b06-419c-9de4-dec615374497-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c547695c-7b06-419c-9de4-dec615374497" (UID: "c547695c-7b06-419c-9de4-dec615374497"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:28:31.765882 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.765853    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-istio-token" (OuterVolumeSpecName: "istio-token") pod "c547695c-7b06-419c-9de4-dec615374497" (UID: "c547695c-7b06-419c-9de4-dec615374497"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:31.864348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.864307    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:31.864348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.864346    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:31.864348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.864354    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c547695c-7b06-419c-9de4-dec615374497-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:31.864348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.864364    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:31.864613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.864372    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:31.864613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.864380    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsmzn\" (UniqueName: \"kubernetes.io/projected/c547695c-7b06-419c-9de4-dec615374497-kube-api-access-qsmzn\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:31.864613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.864390    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c547695c-7b06-419c-9de4-dec615374497-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:31.864613 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:31.864398    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c547695c-7b06-419c-9de4-dec615374497-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:32.457798 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.457751    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" event={"ID":"c547695c-7b06-419c-9de4-dec615374497","Type":"ContainerDied","Data":"a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413"}
Sep 10 21:28:32.458623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.457731    2573 generic.go:356] "Generic (PLEG): container finished" podID="c547695c-7b06-419c-9de4-dec615374497" containerID="a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413" exitCode=0
Sep 10 21:28:32.458623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.457993    2573 scope.go:117] "RemoveContainer" containerID="a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413"
Sep 10 21:28:32.458623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.458234    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm" event={"ID":"c547695c-7b06-419c-9de4-dec615374497","Type":"ContainerDied","Data":"3b6e0475270866477ab9aea72313c1a42ad343e8da208249a7e2013fa7e7f0a1"}
Sep 10 21:28:32.458623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.458263    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"
Sep 10 21:28:32.474245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.474225    2573 scope.go:117] "RemoveContainer" containerID="b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904"
Sep 10 21:28:32.482659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.482616    2573 scope.go:117] "RemoveContainer" containerID="a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413"
Sep 10 21:28:32.482996 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:28:32.482975    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413\": container with ID starting with a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413 not found: ID does not exist" containerID="a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413"
Sep 10 21:28:32.483095 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.483007    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413"} err="failed to get container status \"a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413\": rpc error: code = NotFound desc = could not find container \"a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413\": container with ID starting with a314357c30bf753040d2511edc4561781f55ba0b879eec31e198fe9f50472413 not found: ID does not exist"
Sep 10 21:28:32.483095 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.483031    2573 scope.go:117] "RemoveContainer" containerID="b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904"
Sep 10 21:28:32.483304 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:28:32.483287    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904\": container with ID starting with b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904 not found: ID does not exist" containerID="b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904"
Sep 10 21:28:32.483356 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.483310    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904"} err="failed to get container status \"b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904\": rpc error: code = NotFound desc = could not find container \"b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904\": container with ID starting with b14e6aecc437524ff17a852da401bd6914353386ccf790d8ff051d3634670904 not found: ID does not exist"
Sep 10 21:28:32.484955 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.484925    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"]
Sep 10 21:28:32.490645 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.490625    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a6441b-openshift-default-757bddc5fb-qfwbm"]
Sep 10 21:28:32.679325 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:32.679292    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c547695c-7b06-419c-9de4-dec615374497" path="/var/lib/kubelet/pods/c547695c-7b06-419c-9de4-dec615374497/volumes"
Sep 10 21:28:34.196207 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:34.196172    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"]
Sep 10 21:28:34.196711 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:34.196403    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy" containerID="cri-o://1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5" gracePeriod=30
Sep 10 21:28:39.445633 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.445601    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:28:39.499930 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.499842    2573 generic.go:356] "Generic (PLEG): container finished" podID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerID="1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5" exitCode=0
Sep 10 21:28:39.499930 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.499896    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"
Sep 10 21:28:39.500150 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.499935    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" event={"ID":"4e5e4128-9038-4d8e-9af9-24bc5920abd7","Type":"ContainerDied","Data":"1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5"}
Sep 10 21:28:39.500150 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.500013    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464" event={"ID":"4e5e4128-9038-4d8e-9af9-24bc5920abd7","Type":"ContainerDied","Data":"6303f1296e127533a1fa473d323178700f2ef2a0b30320e7bf6d0aea2fb1c5e4"}
Sep 10 21:28:39.500150 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.500039    2573 scope.go:117] "RemoveContainer" containerID="1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5"
Sep 10 21:28:39.509012 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.508991    2573 scope.go:117] "RemoveContainer" containerID="58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967"
Sep 10 21:28:39.517009 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.516993    2573 scope.go:117] "RemoveContainer" containerID="1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5"
Sep 10 21:28:39.517268 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:28:39.517252    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5\": container with ID starting with 1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5 not found: ID does not exist" containerID="1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5"
Sep 10 21:28:39.517328 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.517276    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5"} err="failed to get container status \"1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5\": rpc error: code = NotFound desc = could not find container \"1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5\": container with ID starting with 1a91d4690f5ab5927830241fd06498d618b95128072408606f696ede746d07d5 not found: ID does not exist"
Sep 10 21:28:39.517328 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.517293    2573 scope.go:117] "RemoveContainer" containerID="58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967"
Sep 10 21:28:39.517560 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:28:39.517543    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967\": container with ID starting with 58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967 not found: ID does not exist" containerID="58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967"
Sep 10 21:28:39.517611 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.517569    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967"} err="failed to get container status \"58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967\": rpc error: code = NotFound desc = could not find container \"58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967\": container with ID starting with 58abb4b1d2d0924a3a46008810b7ae3e562ee9e4d61f88544c7e3b55e0189967 not found: ID does not exist"
Sep 10 21:28:39.528936 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.528915    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-data\") pod \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") "
Sep 10 21:28:39.529019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.528950    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-credential-socket\") pod \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") "
Sep 10 21:28:39.529019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.528992    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-socket\") pod \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") "
Sep 10 21:28:39.529088 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529024    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istiod-ca-cert\") pod \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") "
Sep 10 21:28:39.529088 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529051    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-podinfo\") pod \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") "
Sep 10 21:28:39.529159 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529094    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42qn8\" (UniqueName: \"kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-kube-api-access-42qn8\") pod \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") "
Sep 10 21:28:39.529159 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529118    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-token\") pod \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") "
Sep 10 21:28:39.529159 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529149    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-envoy\") pod \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") "
Sep 10 21:28:39.529301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529174    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-certs\") pod \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\" (UID: \"4e5e4128-9038-4d8e-9af9-24bc5920abd7\") "
Sep 10 21:28:39.529301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529258    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e5e4128-9038-4d8e-9af9-24bc5920abd7" (UID: "4e5e4128-9038-4d8e-9af9-24bc5920abd7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:39.529301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529275    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e5e4128-9038-4d8e-9af9-24bc5920abd7" (UID: "4e5e4128-9038-4d8e-9af9-24bc5920abd7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:39.529445 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529288    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e5e4128-9038-4d8e-9af9-24bc5920abd7" (UID: "4e5e4128-9038-4d8e-9af9-24bc5920abd7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:39.529497 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529451    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e5e4128-9038-4d8e-9af9-24bc5920abd7" (UID: "4e5e4128-9038-4d8e-9af9-24bc5920abd7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:39.529497 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529466    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e5e4128-9038-4d8e-9af9-24bc5920abd7" (UID: "4e5e4128-9038-4d8e-9af9-24bc5920abd7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:39.529602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529517    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:39.529602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529535    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:39.529602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529548    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:39.529602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.529557    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:39.531439 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.531415    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e5e4128-9038-4d8e-9af9-24bc5920abd7" (UID: "4e5e4128-9038-4d8e-9af9-24bc5920abd7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:39.531439 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.531423    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e5e4128-9038-4d8e-9af9-24bc5920abd7" (UID: "4e5e4128-9038-4d8e-9af9-24bc5920abd7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:28:39.531579 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.531435    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-kube-api-access-42qn8" (OuterVolumeSpecName: "kube-api-access-42qn8") pod "4e5e4128-9038-4d8e-9af9-24bc5920abd7" (UID: "4e5e4128-9038-4d8e-9af9-24bc5920abd7"). InnerVolumeSpecName "kube-api-access-42qn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:39.531579 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.531462    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e5e4128-9038-4d8e-9af9-24bc5920abd7" (UID: "4e5e4128-9038-4d8e-9af9-24bc5920abd7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:39.630170 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.630138    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:39.630170 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.630167    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:39.630170 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.630176    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42qn8\" (UniqueName: \"kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-kube-api-access-42qn8\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:39.630403 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.630186    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:39.630403 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.630195    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e5e4128-9038-4d8e-9af9-24bc5920abd7-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:28:39.820388 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.820356    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"]
Sep 10 21:28:39.825247 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.825213    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-76320447-openshift-default-5bb897b888-ff464"]
Sep 10 21:28:39.907578 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:39.907536    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:28:40.679475 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:40.679444    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" path="/var/lib/kubelet/pods/4e5e4128-9038-4d8e-9af9-24bc5920abd7/volumes"
Sep 10 21:28:48.391262 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391224    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"]
Sep 10 21:28:48.391731 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391704    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy"
Sep 10 21:28:48.391864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391734    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy"
Sep 10 21:28:48.391864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391771    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84820544-974b-46ae-bc3b-4cae7fd97820" containerName="main"
Sep 10 21:28:48.391864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391799    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="84820544-974b-46ae-bc3b-4cae7fd97820" containerName="main"
Sep 10 21:28:48.391864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391825    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6780bcb1-34ff-4dd3-81d1-1888b9d1779b" containerName="main"
Sep 10 21:28:48.391864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391834    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6780bcb1-34ff-4dd3-81d1-1888b9d1779b" containerName="main"
Sep 10 21:28:48.391864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391844    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dd07a8c-df59-480d-982c-f69adc174e72" containerName="maas-api"
Sep 10 21:28:48.391864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391852    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dd07a8c-df59-480d-982c-f69adc174e72" containerName="maas-api"
Sep 10 21:28:48.391864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391862    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy"
Sep 10 21:28:48.392260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391873    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy"
Sep 10 21:28:48.392260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391885    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy"
Sep 10 21:28:48.392260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391893    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy"
Sep 10 21:28:48.392260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.391995    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dd07a8c-df59-480d-982c-f69adc174e72" containerName="maas-api"
Sep 10 21:28:48.392260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.392006    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy"
Sep 10 21:28:48.392260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.392016    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy"
Sep 10 21:28:48.392260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.392028    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="84820544-974b-46ae-bc3b-4cae7fd97820" containerName="main"
Sep 10 21:28:48.392260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.392041    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6780bcb1-34ff-4dd3-81d1-1888b9d1779b" containerName="main"
Sep 10 21:28:48.395466 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.395447    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.397889 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.397868    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 21:28:48.397987 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.397945    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-bd7b8a-openshift-default-dockercfg-4czbr\""
Sep 10 21:28:48.407548 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.407528    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"]
Sep 10 21:28:48.511424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.511378    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-certs\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.511610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.511432    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-socket\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.511610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.511492    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-token\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.511610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.511586    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-data\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.511812 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.511623    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-envoy\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.511812 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.511655    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpqn2\" (UniqueName: \"kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-kube-api-access-rpqn2\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.511812 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.511684    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-credential-socket\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.511940 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.511821    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ab16a8-300d-43ee-8d96-454c3f5788fa-istiod-ca-cert\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.511940 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.511853    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-podinfo\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.612537 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.612498    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-data\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.612537 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.612539    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-envoy\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.612825 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.612568    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpqn2\" (UniqueName: \"kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-kube-api-access-rpqn2\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.612825 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.612592    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-credential-socket\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.612825 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.612621    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ab16a8-300d-43ee-8d96-454c3f5788fa-istiod-ca-cert\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.612825 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.612643    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-podinfo\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.612825 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.612667    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-certs\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.612825 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.612696    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-socket\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.612825 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.612721    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-token\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.613188 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.613088    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-data\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.613188 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.613116    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-credential-socket\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.613342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.613321    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-certs\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.613393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.613337    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-socket\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.613513 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.613490    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ab16a8-300d-43ee-8d96-454c3f5788fa-istiod-ca-cert\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.614904 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.614883    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-envoy\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.615263 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.615240    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-podinfo\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.621792 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.621753    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-token\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.621926 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.621908    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpqn2\" (UniqueName: \"kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-kube-api-access-rpqn2\") pod \"e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") " pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.706539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.706447    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:48.995651 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.995626    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"]
Sep 10 21:28:48.997702 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:28:48.997674    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ab16a8_300d_43ee_8d96_454c3f5788fa.slice/crio-9e8426a5083bcd5ac71de51a55c2c182651e4122d21e1805e8bd06b684a07013 WatchSource:0}: Error finding container 9e8426a5083bcd5ac71de51a55c2c182651e4122d21e1805e8bd06b684a07013: Status 404 returned error can't find the container with id 9e8426a5083bcd5ac71de51a55c2c182651e4122d21e1805e8bd06b684a07013
Sep 10 21:28:48.999936 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.999898    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:28:49.000000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:48.999984    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:28:49.000040 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:49.000028    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:28:49.557630 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:49.557594    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" event={"ID":"85ab16a8-300d-43ee-8d96-454c3f5788fa","Type":"ContainerStarted","Data":"ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc"}
Sep 10 21:28:49.558041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:49.557645    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" event={"ID":"85ab16a8-300d-43ee-8d96-454c3f5788fa","Type":"ContainerStarted","Data":"9e8426a5083bcd5ac71de51a55c2c182651e4122d21e1805e8bd06b684a07013"}
Sep 10 21:28:49.579553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:49.579497    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" podStartSLOduration=1.579480784 podStartE2EDuration="1.579480784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:49.577676099 +0000 UTC m=+1105.511342912" watchObservedRunningTime="2026-09-10 21:28:49.579480784 +0000 UTC m=+1105.513147597"
Sep 10 21:28:49.706757 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:49.706718    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:50.712815 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:50.712759    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:51.575224 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:51.575194    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:51.576280 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:51.576261    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:28:52.659822 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.659758    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"]
Sep 10 21:28:52.660317 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.660289    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy"
Sep 10 21:28:52.660317 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.660315    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy"
Sep 10 21:28:52.660463 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.660405    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c547695c-7b06-419c-9de4-dec615374497" containerName="istio-proxy"
Sep 10 21:28:52.660463 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.660412    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e5e4128-9038-4d8e-9af9-24bc5920abd7" containerName="istio-proxy"
Sep 10 21:28:52.662435 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.662416    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.664837 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.664810    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:28:52.665012 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.664848    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-bd7b8a-serving-cert\""
Sep 10 21:28:52.665443 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.665414    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:28:52.665536 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.665478    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-99rzw\""
Sep 10 21:28:52.665536 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.665421    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:28:52.680464 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.680439    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"]
Sep 10 21:28:52.852617 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.852574    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2plt9\" (UniqueName: \"kubernetes.io/projected/934d789c-6bdc-4722-8430-ddc91cc9495d-kube-api-access-2plt9\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.852804 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.852622    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/934d789c-6bdc-4722-8430-ddc91cc9495d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.852804 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.852738    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.852804 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.852792    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.953979 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.953896    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.953979 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.953932    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.954161 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.953997    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2plt9\" (UniqueName: \"kubernetes.io/projected/934d789c-6bdc-4722-8430-ddc91cc9495d-kube-api-access-2plt9\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.954161 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.954026    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/934d789c-6bdc-4722-8430-ddc91cc9495d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.954698 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.954674    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/934d789c-6bdc-4722-8430-ddc91cc9495d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.956397 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.956366    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.956490 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.956420    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.961702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.961681    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2plt9\" (UniqueName: \"kubernetes.io/projected/934d789c-6bdc-4722-8430-ddc91cc9495d-kube-api-access-2plt9\") pod \"maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:52.973577 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:52.973557    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:53.246279 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:53.246252    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"]
Sep 10 21:28:53.247132 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:28:53.247108    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934d789c_6bdc_4722_8430_ddc91cc9495d.slice/crio-88ff1d8ad00d4d4c926dcda05f97a83e7f89659bbb729726b90a1bd435a8dcd0 WatchSource:0}: Error finding container 88ff1d8ad00d4d4c926dcda05f97a83e7f89659bbb729726b90a1bd435a8dcd0: Status 404 returned error can't find the container with id 88ff1d8ad00d4d4c926dcda05f97a83e7f89659bbb729726b90a1bd435a8dcd0
Sep 10 21:28:53.248932 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:53.248914    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:28:53.587090 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:53.587045    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9" event={"ID":"934d789c-6bdc-4722-8430-ddc91cc9495d","Type":"ContainerStarted","Data":"88ff1d8ad00d4d4c926dcda05f97a83e7f89659bbb729726b90a1bd435a8dcd0"}
Sep 10 21:28:54.594378 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:54.594340    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9" event={"ID":"934d789c-6bdc-4722-8430-ddc91cc9495d","Type":"ContainerStarted","Data":"f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f"}
Sep 10 21:28:54.594762 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:54.594479    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:28:54.612877 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:54.612830    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9" podStartSLOduration=2.26162156 podStartE2EDuration="2.612814112s" totalImagesPullingTime="351.192552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:54.61041677 +0000 UTC m=+1110.544083596" watchObservedRunningTime="2026-09-10 21:28:54.612814112 +0000 UTC m=+1110.546480927"
Sep 10 21:28:55.519704 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.516888    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"]
Sep 10 21:28:55.521105 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.521079    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.524524 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.524498    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-1f3879-openshift-default-dockercfg-ddfqz\""
Sep 10 21:28:55.531483 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.531459    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"]
Sep 10 21:28:55.575993 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.575962    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-token\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.576155 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.576006    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3599639b-5d4d-482f-b00f-24ec11e146e0-istiod-ca-cert\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.576155 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.576031    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-envoy\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.576155 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.576078    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-certs\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.576155 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.576118    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h578\" (UniqueName: \"kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-kube-api-access-2h578\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.576287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.576160    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-podinfo\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.576287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.576187    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-data\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.576287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.576223    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-credential-socket\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.576287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.576275    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-socket\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.676934 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.676896    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3599639b-5d4d-482f-b00f-24ec11e146e0-istiod-ca-cert\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.676934 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.676938    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-envoy\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.676957    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-certs\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677108    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h578\" (UniqueName: \"kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-kube-api-access-2h578\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677169    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-podinfo\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677209    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-data\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677292    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-credential-socket\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677418    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-socket\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677452    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-certs\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677469    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-token\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677503    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-data\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677742    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-credential-socket\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677757    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3599639b-5d4d-482f-b00f-24ec11e146e0-istiod-ca-cert\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.677914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.677807    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-socket\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.679371 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.679350    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-envoy\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.679730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.679712    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-podinfo\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.685127 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.685105    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-token\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.685326 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.685304    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h578\" (UniqueName: \"kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-kube-api-access-2h578\") pod \"e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") " pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:55.833769 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:55.833738    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:56.108720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:56.108692    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"]
Sep 10 21:28:56.110464 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:28:56.110438    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3599639b_5d4d_482f_b00f_24ec11e146e0.slice/crio-c4a0e8c4c144979ab7125e1c72772bc6cf450563fb9e465671bbdfa25b95559a WatchSource:0}: Error finding container c4a0e8c4c144979ab7125e1c72772bc6cf450563fb9e465671bbdfa25b95559a: Status 404 returned error can't find the container with id c4a0e8c4c144979ab7125e1c72772bc6cf450563fb9e465671bbdfa25b95559a
Sep 10 21:28:56.112638 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:56.112608    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:28:56.112707 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:56.112682    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:28:56.112748 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:56.112712    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:28:56.608427 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:56.608393    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" event={"ID":"3599639b-5d4d-482f-b00f-24ec11e146e0","Type":"ContainerStarted","Data":"41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96"}
Sep 10 21:28:56.608638 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:56.608444    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" event={"ID":"3599639b-5d4d-482f-b00f-24ec11e146e0","Type":"ContainerStarted","Data":"c4a0e8c4c144979ab7125e1c72772bc6cf450563fb9e465671bbdfa25b95559a"}
Sep 10 21:28:56.629958 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:56.629880    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" podStartSLOduration=1.629863353 podStartE2EDuration="1.629863353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:56.628138823 +0000 UTC m=+1112.561805638" watchObservedRunningTime="2026-09-10 21:28:56.629863353 +0000 UTC m=+1112.563530166"
Sep 10 21:28:56.834119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:56.834085    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:57.839242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:57.839211    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:58.019297 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:58.019261    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:28:58.620489 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:58.620454    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:28:58.621528 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:28:58.621500    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:29:00.606709 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:00.606676    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:29:07.617016 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:07.616956    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 21:29:07.617505 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:07.617063    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 21:29:08.696399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:08.696367    2573 generic.go:356] "Generic (PLEG): container finished" podID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerID="ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc" exitCode=0
Sep 10 21:29:08.696958 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:08.696535    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" event={"ID":"85ab16a8-300d-43ee-8d96-454c3f5788fa","Type":"ContainerDied","Data":"ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc"}
Sep 10 21:29:08.697083 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:08.697061    2573 scope.go:117] "RemoveContainer" containerID="ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc"
Sep 10 21:29:08.698415 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:08.698378    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:08.698512 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:08.698465    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:08.698551 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:08.698511    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:09.122430 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:09.122388    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:29:09.710174 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:09.709262    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" event={"ID":"85ab16a8-300d-43ee-8d96-454c3f5788fa","Type":"ContainerStarted","Data":"83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed"}
Sep 10 21:29:10.706712 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:10.706676    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:29:10.707048 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:10.707023    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 10 21:29:10.707147 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:10.707083    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 10 21:29:11.707453 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:11.707407    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 10 21:29:11.707878 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:11.707484    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 10 21:29:12.707260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:12.707223    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 10 21:29:12.707434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:12.707295    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 10 21:29:13.707003 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:13.706959    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 10 21:29:13.707468 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:13.707029    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 10 21:29:14.269720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:14.269334    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": read tcp 10.134.0.2:56906->10.134.0.65:15021: read: connection reset by peer" start-of-body=
Sep 10 21:29:14.269720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:14.269397    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": read tcp 10.134.0.2:56906->10.134.0.65:15021: read: connection reset by peer"
Sep 10 21:29:14.707376 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:14.707336    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 10 21:29:14.707894 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:14.707403    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 10 21:29:14.740090 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:14.740058    2573 generic.go:356] "Generic (PLEG): container finished" podID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerID="41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96" exitCode=0
Sep 10 21:29:14.740239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:14.740136    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" event={"ID":"3599639b-5d4d-482f-b00f-24ec11e146e0","Type":"ContainerDied","Data":"41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96"}
Sep 10 21:29:14.740539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:14.740526    2573 scope.go:117] "RemoveContainer" containerID="41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96"
Sep 10 21:29:14.741585 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:14.741557    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:14.741688 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:14.741635    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:14.741688 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:14.741681    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:15.710515 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:15.710482    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:29:15.710929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:15.710750    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:29:15.711350 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:15.711335    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:29:15.748192 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:15.748153    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" event={"ID":"3599639b-5d4d-482f-b00f-24ec11e146e0","Type":"ContainerStarted","Data":"f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1"}
Sep 10 21:29:15.834213 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:15.834168    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:29:15.834412 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:15.834354    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 10 21:29:15.834412 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:15.834401    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 10 21:29:16.834875 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:16.834840    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 10 21:29:16.835258 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:16.834909    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 10 21:29:17.012742 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.006629    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"]
Sep 10 21:29:17.013635 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.013610    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.016293 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.016269    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-5e6835-openshift-default-dockercfg-w7d4g\""
Sep 10 21:29:17.025799 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.025761    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"]
Sep 10 21:29:17.092420 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.092319    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-certs\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.092750 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.092727    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-podinfo\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.093039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.093020    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-token\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.093171 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.093153    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istiod-ca-cert\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.093337 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.093320    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-envoy\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.093480 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.093465    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-credential-socket\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.093630 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.093615    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkblp\" (UniqueName: \"kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-kube-api-access-pkblp\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.093760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.093746    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-data\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.093918 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.093903    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-socket\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.195184 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.195133    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-podinfo\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.195395 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.195197    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-token\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.195395 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.195218    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istiod-ca-cert\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.195395 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.195260    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-envoy\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.195395 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.195286    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-credential-socket\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.195395 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.195314    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkblp\" (UniqueName: \"kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-kube-api-access-pkblp\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.195395 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.195341    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-data\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.195395 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.195364    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-socket\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.195766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.195421    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-certs\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.195964 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.195940    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-certs\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.196243 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.196217    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-data\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.196557 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.196530    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-socket\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.196648 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.196609    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-credential-socket\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.196710 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.196658    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istiod-ca-cert\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.198503 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.198481    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-envoy\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.198586 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.198556    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-podinfo\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.203510 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.203486    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-token\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.203933 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.203917    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkblp\" (UniqueName: \"kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-kube-api-access-pkblp\") pod \"e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") " pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.325716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.325675    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:17.629568 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.629540    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"]
Sep 10 21:29:17.630676 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:29:17.630646    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b9a14d_d6cb_405b_bbd1_cce61559fbb4.slice/crio-5e70ef920e44d140b2c9c79d6aa3bbe64a13f2184ea57aa3144f3f3668456e9c WatchSource:0}: Error finding container 5e70ef920e44d140b2c9c79d6aa3bbe64a13f2184ea57aa3144f3f3668456e9c: Status 404 returned error can't find the container with id 5e70ef920e44d140b2c9c79d6aa3bbe64a13f2184ea57aa3144f3f3668456e9c
Sep 10 21:29:17.633168 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.633131    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:17.633244 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.633214    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:17.633285 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.633254    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:17.762716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.762673    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" event={"ID":"96b9a14d-d6cb-405b-bbd1-cce61559fbb4","Type":"ContainerStarted","Data":"7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c"}
Sep 10 21:29:17.762919 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.762745    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" event={"ID":"96b9a14d-d6cb-405b-bbd1-cce61559fbb4","Type":"ContainerStarted","Data":"5e70ef920e44d140b2c9c79d6aa3bbe64a13f2184ea57aa3144f3f3668456e9c"}
Sep 10 21:29:17.834506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.834470    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 10 21:29:17.834668 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:17.834521    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 10 21:29:18.325879 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:18.325843    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:18.330879 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:18.330850    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:18.356667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:18.356616    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" podStartSLOduration=2.3565994679999998 podStartE2EDuration="2.356599468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:17.788602261 +0000 UTC m=+1133.722269089" watchObservedRunningTime="2026-09-10 21:29:18.356599468 +0000 UTC m=+1134.290266283"
Sep 10 21:29:18.768850 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:18.768750    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:18.769619 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:18.769603    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:18.834434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:18.834397    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 10 21:29:18.834601 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:18.834448    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 10 21:29:19.834879 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:19.834842    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 10 21:29:19.835255 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:19.834896    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 10 21:29:20.837846 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:20.837817    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:29:20.838229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:20.838167    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:29:20.838836 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:20.838818    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:29:27.021290 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:27.021250    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:29:34.809946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:34.809901    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 21:29:34.810453 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:34.809971    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 21:29:35.814853 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.814821    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"]
Sep 10 21:29:35.817355 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.817337    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.820240 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.820213    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-36b938\"/\"default-dockercfg-w26pp\""
Sep 10 21:29:35.820240 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.820213    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-36b938\"/\"openshift-service-ca.crt\""
Sep 10 21:29:35.820458 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.820261    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-36b938\"/\"kube-root-ca.crt\""
Sep 10 21:29:35.820458 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.820282    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-36b938\"/\"auth-test-model-36b938-kserve-self-signed-certs\""
Sep 10 21:29:35.826878 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.826852    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"]
Sep 10 21:29:35.879821 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.879744    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-home\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.880000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.879927    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-tmp-dir\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.880000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.879977    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-dshm\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.880133 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.880071    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-model-cache\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.880133 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.880101    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a681998-2116-437f-831a-e26a564490d7-tls-certs\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.880250 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.880155    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24hk7\" (UniqueName: \"kubernetes.io/projected/7a681998-2116-437f-831a-e26a564490d7-kube-api-access-24hk7\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.981646 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.981605    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-home\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.981857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.981668    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-tmp-dir\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.981857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.981699    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-dshm\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.981857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.981752    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-model-cache\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.982019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.981887    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a681998-2116-437f-831a-e26a564490d7-tls-certs\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.982019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.981958    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24hk7\" (UniqueName: \"kubernetes.io/projected/7a681998-2116-437f-831a-e26a564490d7-kube-api-access-24hk7\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.982131 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.982083    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-tmp-dir\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.982131 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.982106    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-model-cache\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.982229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.982178    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-home\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.984026 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.984001    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-dshm\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.984293 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.984277    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a681998-2116-437f-831a-e26a564490d7-tls-certs\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:35.995123 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:35.995098    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24hk7\" (UniqueName: \"kubernetes.io/projected/7a681998-2116-437f-831a-e26a564490d7-kube-api-access-24hk7\") pod \"auth-test-model-36b938-kserve-78ff7b7687-w8rlg\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") " pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:36.131075 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.131037    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:36.407379 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.407352    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"]
Sep 10 21:29:36.410061 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:29:36.410031    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a681998_2116_437f_831a_e26a564490d7.slice/crio-146a59aa781c3f6f0dc96a08cfa9230c9535d787d2e6ac81c8b1d4a91cec3f2c WatchSource:0}: Error finding container 146a59aa781c3f6f0dc96a08cfa9230c9535d787d2e6ac81c8b1d4a91cec3f2c: Status 404 returned error can't find the container with id 146a59aa781c3f6f0dc96a08cfa9230c9535d787d2e6ac81c8b1d4a91cec3f2c
Sep 10 21:29:36.880670 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.880640    2573 generic.go:356] "Generic (PLEG): container finished" podID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerID="7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c" exitCode=0
Sep 10 21:29:36.881185 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.880721    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" event={"ID":"96b9a14d-d6cb-405b-bbd1-cce61559fbb4","Type":"ContainerDied","Data":"7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c"}
Sep 10 21:29:36.881273 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.881249    2573 scope.go:117] "RemoveContainer" containerID="7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c"
Sep 10 21:29:36.882409 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.882376    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:36.882504 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.882465    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:36.882559 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.882513    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:36.882712 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.882692    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg" event={"ID":"7a681998-2116-437f-831a-e26a564490d7","Type":"ContainerStarted","Data":"bec9fd65b39d0fb212d2a0ff950e3a6357e1445b2afd597266ec5dfd731c1720"}
Sep 10 21:29:36.882760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.882722    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg" event={"ID":"7a681998-2116-437f-831a-e26a564490d7","Type":"ContainerStarted","Data":"146a59aa781c3f6f0dc96a08cfa9230c9535d787d2e6ac81c8b1d4a91cec3f2c"}
Sep 10 21:29:36.917031 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:36.916978    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg" podStartSLOduration=1.916947778 podStartE2EDuration="1.916947778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:36.914104313 +0000 UTC m=+1152.847771138" watchObservedRunningTime="2026-09-10 21:29:36.916947778 +0000 UTC m=+1152.850614587"
Sep 10 21:29:37.896891 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:37.896853    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" event={"ID":"96b9a14d-d6cb-405b-bbd1-cce61559fbb4","Type":"ContainerStarted","Data":"5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d"}
Sep 10 21:29:38.326301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:38.326259    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:38.326551 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:38.326527    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 10 21:29:38.326634 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:38.326583    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 10 21:29:39.326638 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:39.326595    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 10 21:29:39.327130 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:39.326690    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 10 21:29:40.326207 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:40.326168    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 10 21:29:40.326394 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:40.326225    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 10 21:29:40.615473 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:40.615375    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:29:41.327134 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:41.327097    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 10 21:29:41.327326 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:41.327158    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 10 21:29:42.326978 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:42.326944    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 10 21:29:42.327356 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:42.327001    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 10 21:29:43.402390 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:43.402356    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:43.402839 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:43.402746    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:43.403545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:43.403529    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:29:45.398743 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:45.398701    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7791-94b47a"]
Sep 10 21:29:45.401808 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:45.401768    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-94b47a"
Sep 10 21:29:45.404052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:45.404030    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dxcl2\""
Sep 10 21:29:45.410016 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:45.409453    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-94b47a"]
Sep 10 21:29:45.480931 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:45.480895    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2tgr\" (UniqueName: \"kubernetes.io/projected/dbcff82d-0374-45bc-a1a6-1b0d6387272a-kube-api-access-s2tgr\") pod \"test-curl-7791-94b47a\" (UID: \"dbcff82d-0374-45bc-a1a6-1b0d6387272a\") " pod="opendatahub/test-curl-7791-94b47a"
Sep 10 21:29:45.582309 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:45.582270    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2tgr\" (UniqueName: \"kubernetes.io/projected/dbcff82d-0374-45bc-a1a6-1b0d6387272a-kube-api-access-s2tgr\") pod \"test-curl-7791-94b47a\" (UID: \"dbcff82d-0374-45bc-a1a6-1b0d6387272a\") " pod="opendatahub/test-curl-7791-94b47a"
Sep 10 21:29:45.590238 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:45.590202    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2tgr\" (UniqueName: \"kubernetes.io/projected/dbcff82d-0374-45bc-a1a6-1b0d6387272a-kube-api-access-s2tgr\") pod \"test-curl-7791-94b47a\" (UID: \"dbcff82d-0374-45bc-a1a6-1b0d6387272a\") " pod="opendatahub/test-curl-7791-94b47a"
Sep 10 21:29:45.713689 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:45.713595    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-94b47a"
Sep 10 21:29:45.980398 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:45.980379    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-94b47a"]
Sep 10 21:29:45.982805 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:29:45.982763    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbcff82d_0374_45bc_a1a6_1b0d6387272a.slice/crio-d3c28e967989d46e6fc86fbf249111e2475eb2a93e2acf924a7565f5ad2a8e0f WatchSource:0}: Error finding container d3c28e967989d46e6fc86fbf249111e2475eb2a93e2acf924a7565f5ad2a8e0f: Status 404 returned error can't find the container with id d3c28e967989d46e6fc86fbf249111e2475eb2a93e2acf924a7565f5ad2a8e0f
Sep 10 21:29:46.131668 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:46.131636    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:46.131884 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:46.131683    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:46.143948 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:46.143920    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:46.959830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:46.959697    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-94b47a" event={"ID":"dbcff82d-0374-45bc-a1a6-1b0d6387272a","Type":"ContainerStarted","Data":"b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4"}
Sep 10 21:29:46.959830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:46.959770    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-94b47a" event={"ID":"dbcff82d-0374-45bc-a1a6-1b0d6387272a","Type":"ContainerStarted","Data":"d3c28e967989d46e6fc86fbf249111e2475eb2a93e2acf924a7565f5ad2a8e0f"}
Sep 10 21:29:46.970858 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:46.970835    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:29:46.973420 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:46.973374    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7791-94b47a" podStartSLOduration=1.9733617030000001 podStartE2EDuration="1.973361703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:46.972173356 +0000 UTC m=+1162.905840171" watchObservedRunningTime="2026-09-10 21:29:46.973361703 +0000 UTC m=+1162.907028517"
Sep 10 21:29:47.559236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.559201    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7791-94b47a"]
Sep 10 21:29:47.563748 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.563710    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7791-94b47a"]
Sep 10 21:29:47.681522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.681445    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7791-2c3066"]
Sep 10 21:29:47.681824 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.681811    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbcff82d-0374-45bc-a1a6-1b0d6387272a" containerName="test-curl-7791-94b47a"
Sep 10 21:29:47.681873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.681824    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbcff82d-0374-45bc-a1a6-1b0d6387272a" containerName="test-curl-7791-94b47a"
Sep 10 21:29:47.681909 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.681892    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbcff82d-0374-45bc-a1a6-1b0d6387272a" containerName="test-curl-7791-94b47a"
Sep 10 21:29:47.683846 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.683830    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-2c3066"
Sep 10 21:29:47.691247 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.691222    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-2c3066"]
Sep 10 21:29:47.803018 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.802981    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjs54\" (UniqueName: \"kubernetes.io/projected/a5204697-35ba-46ec-8c43-d9d07a67cdb4-kube-api-access-mjs54\") pod \"test-curl-7791-2c3066\" (UID: \"a5204697-35ba-46ec-8c43-d9d07a67cdb4\") " pod="opendatahub/test-curl-7791-2c3066"
Sep 10 21:29:47.903728 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.903684    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjs54\" (UniqueName: \"kubernetes.io/projected/a5204697-35ba-46ec-8c43-d9d07a67cdb4-kube-api-access-mjs54\") pod \"test-curl-7791-2c3066\" (UID: \"a5204697-35ba-46ec-8c43-d9d07a67cdb4\") " pod="opendatahub/test-curl-7791-2c3066"
Sep 10 21:29:47.911997 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.911972    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjs54\" (UniqueName: \"kubernetes.io/projected/a5204697-35ba-46ec-8c43-d9d07a67cdb4-kube-api-access-mjs54\") pod \"test-curl-7791-2c3066\" (UID: \"a5204697-35ba-46ec-8c43-d9d07a67cdb4\") " pod="opendatahub/test-curl-7791-2c3066"
Sep 10 21:29:47.965631 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:47.965544    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7791-94b47a" container="test-curl-7791-94b47a"
Sep 10 21:29:47.993928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:47.993908    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-2c3066"
Sep 10 21:29:48.263540 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:48.263511    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-2c3066"]
Sep 10 21:29:48.265199 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:29:48.265165    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5204697_35ba_46ec_8c43_d9d07a67cdb4.slice/crio-5e1b6cb52b46b9c7bf3691b2030ddf37f2c39cb5908bd9cbad2de6baf245785b WatchSource:0}: Error finding container 5e1b6cb52b46b9c7bf3691b2030ddf37f2c39cb5908bd9cbad2de6baf245785b: Status 404 returned error can't find the container with id 5e1b6cb52b46b9c7bf3691b2030ddf37f2c39cb5908bd9cbad2de6baf245785b
Sep 10 21:29:48.973251 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:48.973207    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-2c3066" event={"ID":"a5204697-35ba-46ec-8c43-d9d07a67cdb4","Type":"ContainerStarted","Data":"3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618"}
Sep 10 21:29:48.973251 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:48.973256    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-2c3066" event={"ID":"a5204697-35ba-46ec-8c43-d9d07a67cdb4","Type":"ContainerStarted","Data":"5e1b6cb52b46b9c7bf3691b2030ddf37f2c39cb5908bd9cbad2de6baf245785b"}
Sep 10 21:29:48.973738 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:48.973261    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7791-94b47a" podUID="dbcff82d-0374-45bc-a1a6-1b0d6387272a" containerName="test-curl-7791-94b47a" containerID="cri-o://b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4" gracePeriod=2
Sep 10 21:29:48.990451 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:48.990405    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7791-2c3066" podStartSLOduration=1.990390863 podStartE2EDuration="1.990390863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:48.988884568 +0000 UTC m=+1164.922551384" watchObservedRunningTime="2026-09-10 21:29:48.990390863 +0000 UTC m=+1164.924057677"
Sep 10 21:29:49.582914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.582874    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7791-2c3066"]
Sep 10 21:29:49.587614 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.587583    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7791-2c3066"]
Sep 10 21:29:49.701815 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.701707    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7791-7922f0"]
Sep 10 21:29:49.702188 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.702169    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5204697-35ba-46ec-8c43-d9d07a67cdb4" containerName="test-curl-7791-2c3066"
Sep 10 21:29:49.702275 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.702191    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5204697-35ba-46ec-8c43-d9d07a67cdb4" containerName="test-curl-7791-2c3066"
Sep 10 21:29:49.702327 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.702293    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5204697-35ba-46ec-8c43-d9d07a67cdb4" containerName="test-curl-7791-2c3066"
Sep 10 21:29:49.704558 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.704536    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-7922f0"
Sep 10 21:29:49.710531 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.710508    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-7922f0"]
Sep 10 21:29:49.822282 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.822242    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmhzk\" (UniqueName: \"kubernetes.io/projected/43c9efd7-125b-469c-9e9a-c396f148a188-kube-api-access-pmhzk\") pod \"test-curl-7791-7922f0\" (UID: \"43c9efd7-125b-469c-9e9a-c396f148a188\") " pod="opendatahub/test-curl-7791-7922f0"
Sep 10 21:29:49.923177 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.923144    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhzk\" (UniqueName: \"kubernetes.io/projected/43c9efd7-125b-469c-9e9a-c396f148a188-kube-api-access-pmhzk\") pod \"test-curl-7791-7922f0\" (UID: \"43c9efd7-125b-469c-9e9a-c396f148a188\") " pod="opendatahub/test-curl-7791-7922f0"
Sep 10 21:29:49.931994 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:49.931967    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmhzk\" (UniqueName: \"kubernetes.io/projected/43c9efd7-125b-469c-9e9a-c396f148a188-kube-api-access-pmhzk\") pod \"test-curl-7791-7922f0\" (UID: \"43c9efd7-125b-469c-9e9a-c396f148a188\") " pod="opendatahub/test-curl-7791-7922f0"
Sep 10 21:29:49.978461 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:49.978377    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7791-2c3066" container="test-curl-7791-2c3066"
Sep 10 21:29:50.014787 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:50.014754    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-7922f0"
Sep 10 21:29:50.281243 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:50.281217    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-7922f0"]
Sep 10 21:29:50.282955 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:29:50.282925    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43c9efd7_125b_469c_9e9a_c396f148a188.slice/crio-aa0c9d333ddc1653c237841a0d392da2878daa75cf8ebd849e46e29f5667782a WatchSource:0}: Error finding container aa0c9d333ddc1653c237841a0d392da2878daa75cf8ebd849e46e29f5667782a: Status 404 returned error can't find the container with id aa0c9d333ddc1653c237841a0d392da2878daa75cf8ebd849e46e29f5667782a
Sep 10 21:29:50.986205 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:50.986173    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-7922f0" event={"ID":"43c9efd7-125b-469c-9e9a-c396f148a188","Type":"ContainerStarted","Data":"ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10"}
Sep 10 21:29:50.986549 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:50.986214    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-7922f0" event={"ID":"43c9efd7-125b-469c-9e9a-c396f148a188","Type":"ContainerStarted","Data":"aa0c9d333ddc1653c237841a0d392da2878daa75cf8ebd849e46e29f5667782a"}
Sep 10 21:29:50.986549 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:50.986328    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7791-2c3066" podUID="a5204697-35ba-46ec-8c43-d9d07a67cdb4" containerName="test-curl-7791-2c3066" containerID="cri-o://3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618" gracePeriod=2
Sep 10 21:29:51.000597 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.000557    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7791-7922f0" podStartSLOduration=2.000542804 podStartE2EDuration="2.000542804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:50.998652464 +0000 UTC m=+1166.932319277" watchObservedRunningTime="2026-09-10 21:29:51.000542804 +0000 UTC m=+1166.934209620"
Sep 10 21:29:51.133710 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.133685    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-94b47a"
Sep 10 21:29:51.236817 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.236728    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2tgr\" (UniqueName: \"kubernetes.io/projected/dbcff82d-0374-45bc-a1a6-1b0d6387272a-kube-api-access-s2tgr\") pod \"dbcff82d-0374-45bc-a1a6-1b0d6387272a\" (UID: \"dbcff82d-0374-45bc-a1a6-1b0d6387272a\") "
Sep 10 21:29:51.238602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.238577    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbcff82d-0374-45bc-a1a6-1b0d6387272a-kube-api-access-s2tgr" (OuterVolumeSpecName: "kube-api-access-s2tgr") pod "dbcff82d-0374-45bc-a1a6-1b0d6387272a" (UID: "dbcff82d-0374-45bc-a1a6-1b0d6387272a"). InnerVolumeSpecName "kube-api-access-s2tgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:29:51.337563 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.337516    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2tgr\" (UniqueName: \"kubernetes.io/projected/dbcff82d-0374-45bc-a1a6-1b0d6387272a-kube-api-access-s2tgr\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:29:51.577912 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.577872    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7791-7922f0"]
Sep 10 21:29:51.581889 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.581859    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7791-7922f0"]
Sep 10 21:29:51.691680 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.691649    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7791-a54104"]
Sep 10 21:29:51.692115 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.692101    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c9efd7-125b-469c-9e9a-c396f148a188" containerName="test-curl-7791-7922f0"
Sep 10 21:29:51.692162 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.692117    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c9efd7-125b-469c-9e9a-c396f148a188" containerName="test-curl-7791-7922f0"
Sep 10 21:29:51.692206 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.692189    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c9efd7-125b-469c-9e9a-c396f148a188" containerName="test-curl-7791-7922f0"
Sep 10 21:29:51.694213 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.694197    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-a54104"
Sep 10 21:29:51.699958 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.699931    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-a54104"]
Sep 10 21:29:51.742053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.742019    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgrfn\" (UniqueName: \"kubernetes.io/projected/36b79698-3da3-4fa8-bb26-34481b34daaa-kube-api-access-xgrfn\") pod \"test-curl-7791-a54104\" (UID: \"36b79698-3da3-4fa8-bb26-34481b34daaa\") " pod="opendatahub/test-curl-7791-a54104"
Sep 10 21:29:51.843299 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.843223    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgrfn\" (UniqueName: \"kubernetes.io/projected/36b79698-3da3-4fa8-bb26-34481b34daaa-kube-api-access-xgrfn\") pod \"test-curl-7791-a54104\" (UID: \"36b79698-3da3-4fa8-bb26-34481b34daaa\") " pod="opendatahub/test-curl-7791-a54104"
Sep 10 21:29:51.851688 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.851661    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgrfn\" (UniqueName: \"kubernetes.io/projected/36b79698-3da3-4fa8-bb26-34481b34daaa-kube-api-access-xgrfn\") pod \"test-curl-7791-a54104\" (UID: \"36b79698-3da3-4fa8-bb26-34481b34daaa\") " pod="opendatahub/test-curl-7791-a54104"
Sep 10 21:29:51.993374 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.993333    2573 generic.go:356] "Generic (PLEG): container finished" podID="dbcff82d-0374-45bc-a1a6-1b0d6387272a" containerID="b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4" exitCode=137
Sep 10 21:29:51.993766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.993384    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-94b47a"
Sep 10 21:29:51.993766 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:51.993677    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7791-7922f0" container="test-curl-7791-7922f0"
Sep 10 21:29:51.994392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:51.993443    2573 scope.go:117] "RemoveContainer" containerID="b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4"
Sep 10 21:29:52.003497 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:52.003482    2573 scope.go:117] "RemoveContainer" containerID="b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4"
Sep 10 21:29:52.003813 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:52.003753    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4\": container with ID starting with b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4 not found: ID does not exist" containerID="b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4"
Sep 10 21:29:52.003947 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:52.003823    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4"} err="failed to get container status \"b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4\": rpc error: code = NotFound desc = could not find container \"b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4\": container with ID starting with b9534bb9d16a16e6593cebf1332cd7d0b4895ac69e5b8a00fc48e335441668b4 not found: ID does not exist"
Sep 10 21:29:52.003947 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:52.003919    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7791-94b47a\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="dbcff82d-0374-45bc-a1a6-1b0d6387272a" pod="opendatahub/test-curl-7791-94b47a"
Sep 10 21:29:52.015510 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:52.015489    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-a54104"
Sep 10 21:29:52.285641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:52.285609    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-a54104"]
Sep 10 21:29:52.286877 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:29:52.286848    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36b79698_3da3_4fa8_bb26_34481b34daaa.slice/crio-aa82aad8b1e17c50a64ae3011ca9f451558b956670c14cc5b2d297fc0cb3ccb5 WatchSource:0}: Error finding container aa82aad8b1e17c50a64ae3011ca9f451558b956670c14cc5b2d297fc0cb3ccb5: Status 404 returned error can't find the container with id aa82aad8b1e17c50a64ae3011ca9f451558b956670c14cc5b2d297fc0cb3ccb5
Sep 10 21:29:52.679616 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:52.679575    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbcff82d-0374-45bc-a1a6-1b0d6387272a" path="/var/lib/kubelet/pods/dbcff82d-0374-45bc-a1a6-1b0d6387272a/volumes"
Sep 10 21:29:53.003020 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:53.002995    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-a54104" event={"ID":"36b79698-3da3-4fa8-bb26-34481b34daaa","Type":"ContainerStarted","Data":"9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370"}
Sep 10 21:29:53.003292 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:53.003033    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-a54104" event={"ID":"36b79698-3da3-4fa8-bb26-34481b34daaa","Type":"ContainerStarted","Data":"aa82aad8b1e17c50a64ae3011ca9f451558b956670c14cc5b2d297fc0cb3ccb5"}
Sep 10 21:29:53.003292 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:53.003098    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7791-7922f0" podUID="43c9efd7-125b-469c-9e9a-c396f148a188" containerName="test-curl-7791-7922f0" containerID="cri-o://ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10" gracePeriod=2
Sep 10 21:29:53.023770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:53.023702    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7791-a54104" podStartSLOduration=2.023687929 podStartE2EDuration="2.023687929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:53.022326154 +0000 UTC m=+1168.955992970" watchObservedRunningTime="2026-09-10 21:29:53.023687929 +0000 UTC m=+1168.957354742"
Sep 10 21:29:53.127285 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:53.127260    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-2c3066"
Sep 10 21:29:53.259033 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:53.258956    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjs54\" (UniqueName: \"kubernetes.io/projected/a5204697-35ba-46ec-8c43-d9d07a67cdb4-kube-api-access-mjs54\") pod \"a5204697-35ba-46ec-8c43-d9d07a67cdb4\" (UID: \"a5204697-35ba-46ec-8c43-d9d07a67cdb4\") "
Sep 10 21:29:53.260945 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:53.260914    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5204697-35ba-46ec-8c43-d9d07a67cdb4-kube-api-access-mjs54" (OuterVolumeSpecName: "kube-api-access-mjs54") pod "a5204697-35ba-46ec-8c43-d9d07a67cdb4" (UID: "a5204697-35ba-46ec-8c43-d9d07a67cdb4"). InnerVolumeSpecName "kube-api-access-mjs54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:29:53.360604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:53.360573    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjs54\" (UniqueName: \"kubernetes.io/projected/a5204697-35ba-46ec-8c43-d9d07a67cdb4-kube-api-access-mjs54\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:29:53.598384 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:53.598350    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7791-a54104"]
Sep 10 21:29:53.602739 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:53.602712    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7791-a54104"]
Sep 10 21:29:54.009871 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.009771    2573 generic.go:356] "Generic (PLEG): container finished" podID="a5204697-35ba-46ec-8c43-d9d07a67cdb4" containerID="3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618" exitCode=137
Sep 10 21:29:54.009871 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.009837    2573 scope.go:117] "RemoveContainer" containerID="3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618"
Sep 10 21:29:54.009871 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.009844    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-2c3066"
Sep 10 21:29:54.010452 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:54.010209    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7791-a54104" container="test-curl-7791-a54104"
Sep 10 21:29:54.019328 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.019312    2573 scope.go:117] "RemoveContainer" containerID="3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618"
Sep 10 21:29:54.019611 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:54.019594    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618\": container with ID starting with 3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618 not found: ID does not exist" containerID="3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618"
Sep 10 21:29:54.019655 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.019621    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618"} err="failed to get container status \"3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618\": rpc error: code = NotFound desc = could not find container \"3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618\": container with ID starting with 3e1d576b3f6e2f9f01aae23142b6658e5ad784ccc02f1355dce20f73330f9618 not found: ID does not exist"
Sep 10 21:29:54.019907 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:54.019887    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7791-2c3066\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="a5204697-35ba-46ec-8c43-d9d07a67cdb4" pod="opendatahub/test-curl-7791-2c3066"
Sep 10 21:29:54.450531 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.450497    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"]
Sep 10 21:29:54.451142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.451121    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36b79698-3da3-4fa8-bb26-34481b34daaa" containerName="test-curl-7791-a54104"
Sep 10 21:29:54.451212 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.451146    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="36b79698-3da3-4fa8-bb26-34481b34daaa" containerName="test-curl-7791-a54104"
Sep 10 21:29:54.451259 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.451245    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="36b79698-3da3-4fa8-bb26-34481b34daaa" containerName="test-curl-7791-a54104"
Sep 10 21:29:54.454336 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.454314    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.456185 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:54.456151    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7791-2c3066\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="a5204697-35ba-46ec-8c43-d9d07a67cdb4" pod="opendatahub/test-curl-7791-2c3066"
Sep 10 21:29:54.456687 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.456670    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7ce89dd6-openshift-default-dockercfg-qkmmj\""
Sep 10 21:29:54.466161 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.466141    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"]
Sep 10 21:29:54.571260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.571220    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t22kp\" (UniqueName: \"kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-kube-api-access-t22kp\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.571260 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.571264    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-token\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.571470 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.571349    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-podinfo\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.571470 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.571403    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-data\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.571470 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.571425    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-envoy\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.571470 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.571448    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ddc4d1-72bc-48f4-974a-6311231611e1-istiod-ca-cert\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.571602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.571472    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-credential-socket\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.571602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.571494    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-socket\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.571602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.571514    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-certs\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.672322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672280    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-podinfo\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.672526 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672338    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-data\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.672526 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672364    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-envoy\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.672526 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672392    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ddc4d1-72bc-48f4-974a-6311231611e1-istiod-ca-cert\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.672526 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672428    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-credential-socket\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.672526 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672450    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-socket\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.672526 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672474    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-certs\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.672885 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672593    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t22kp\" (UniqueName: \"kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-kube-api-access-t22kp\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.672885 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672621    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-token\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.673005 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672881    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-data\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.673005 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.672945    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-credential-socket\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.673344 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.673319    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ddc4d1-72bc-48f4-974a-6311231611e1-istiod-ca-cert\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.673686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.673659    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-certs\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.673835 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.673663    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-socket\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.675176 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.675154    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-podinfo\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.675400 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.675383    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-envoy\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.677786 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:54.677744    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7791-2c3066\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="a5204697-35ba-46ec-8c43-d9d07a67cdb4" pod="opendatahub/test-curl-7791-2c3066"
Sep 10 21:29:54.679821 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.679797    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5204697-35ba-46ec-8c43-d9d07a67cdb4" path="/var/lib/kubelet/pods/a5204697-35ba-46ec-8c43-d9d07a67cdb4/volumes"
Sep 10 21:29:54.681001 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.680977    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-token\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.681099 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.681083    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t22kp\" (UniqueName: \"kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-kube-api-access-t22kp\") pod \"e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") " pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:54.765306 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:54.765223    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:55.020002 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:55.019966    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7791-a54104" podUID="36b79698-3da3-4fa8-bb26-34481b34daaa" containerName="test-curl-7791-a54104" containerID="cri-o://9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370" gracePeriod=2
Sep 10 21:29:55.042663 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:55.042635    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"]
Sep 10 21:29:55.063014 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:29:55.062984    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ddc4d1_72bc_48f4_974a_6311231611e1.slice/crio-4a590dcca846955513673f836f29607356289ea1bf3f2174b3d48811e8bb7d0b WatchSource:0}: Error finding container 4a590dcca846955513673f836f29607356289ea1bf3f2174b3d48811e8bb7d0b: Status 404 returned error can't find the container with id 4a590dcca846955513673f836f29607356289ea1bf3f2174b3d48811e8bb7d0b
Sep 10 21:29:55.065331 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:55.065298    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:55.065419 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:55.065362    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:55.065419 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:55.065388    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:29:55.152168 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:55.152139    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-7922f0"
Sep 10 21:29:55.279002 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:55.278915    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmhzk\" (UniqueName: \"kubernetes.io/projected/43c9efd7-125b-469c-9e9a-c396f148a188-kube-api-access-pmhzk\") pod \"43c9efd7-125b-469c-9e9a-c396f148a188\" (UID: \"43c9efd7-125b-469c-9e9a-c396f148a188\") "
Sep 10 21:29:55.281058 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:55.281029    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c9efd7-125b-469c-9e9a-c396f148a188-kube-api-access-pmhzk" (OuterVolumeSpecName: "kube-api-access-pmhzk") pod "43c9efd7-125b-469c-9e9a-c396f148a188" (UID: "43c9efd7-125b-469c-9e9a-c396f148a188"). InnerVolumeSpecName "kube-api-access-pmhzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:29:55.380187 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:55.380149    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmhzk\" (UniqueName: \"kubernetes.io/projected/43c9efd7-125b-469c-9e9a-c396f148a188-kube-api-access-pmhzk\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:29:56.027087 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.026988    2573 generic.go:356] "Generic (PLEG): container finished" podID="43c9efd7-125b-469c-9e9a-c396f148a188" containerID="ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10" exitCode=137
Sep 10 21:29:56.027087 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.027040    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-7922f0"
Sep 10 21:29:56.027576 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.027091    2573 scope.go:117] "RemoveContainer" containerID="ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10"
Sep 10 21:29:56.028684 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.028655    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" event={"ID":"e4ddc4d1-72bc-48f4-974a-6311231611e1","Type":"ContainerStarted","Data":"9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da"}
Sep 10 21:29:56.028793 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.028742    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" event={"ID":"e4ddc4d1-72bc-48f4-974a-6311231611e1","Type":"ContainerStarted","Data":"4a590dcca846955513673f836f29607356289ea1bf3f2174b3d48811e8bb7d0b"}
Sep 10 21:29:56.036435 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.036420    2573 scope.go:117] "RemoveContainer" containerID="ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10"
Sep 10 21:29:56.036686 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:56.036667    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10\": container with ID starting with ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10 not found: ID does not exist" containerID="ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10"
Sep 10 21:29:56.036745 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.036695    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10"} err="failed to get container status \"ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10\": rpc error: code = NotFound desc = could not find container \"ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10\": container with ID starting with ca9d96aff10a0981822ec971d059bd6c72fb2d0f89b51a29e6bfce90203f3b10 not found: ID does not exist"
Sep 10 21:29:56.049679 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:56.049653    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7791-7922f0\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="43c9efd7-125b-469c-9e9a-c396f148a188" pod="opendatahub/test-curl-7791-7922f0"
Sep 10 21:29:56.050552 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.050515    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" podStartSLOduration=2.050503129 podStartE2EDuration="2.050503129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:56.04799814 +0000 UTC m=+1171.981664955" watchObservedRunningTime="2026-09-10 21:29:56.050503129 +0000 UTC m=+1171.984170024"
Sep 10 21:29:56.679352 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.679310    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43c9efd7-125b-469c-9e9a-c396f148a188" path="/var/lib/kubelet/pods/43c9efd7-125b-469c-9e9a-c396f148a188/volumes"
Sep 10 21:29:56.766108 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.766069    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:56.771950 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:56.771922    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:57.038166 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:57.038144    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:57.039182 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:57.039164    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:29:57.177050 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:57.177026    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-a54104"
Sep 10 21:29:57.298659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:57.298562    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgrfn\" (UniqueName: \"kubernetes.io/projected/36b79698-3da3-4fa8-bb26-34481b34daaa-kube-api-access-xgrfn\") pod \"36b79698-3da3-4fa8-bb26-34481b34daaa\" (UID: \"36b79698-3da3-4fa8-bb26-34481b34daaa\") "
Sep 10 21:29:57.300702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:57.300672    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b79698-3da3-4fa8-bb26-34481b34daaa-kube-api-access-xgrfn" (OuterVolumeSpecName: "kube-api-access-xgrfn") pod "36b79698-3da3-4fa8-bb26-34481b34daaa" (UID: "36b79698-3da3-4fa8-bb26-34481b34daaa"). InnerVolumeSpecName "kube-api-access-xgrfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:29:57.399522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:57.399487    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgrfn\" (UniqueName: \"kubernetes.io/projected/36b79698-3da3-4fa8-bb26-34481b34daaa-kube-api-access-xgrfn\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:29:58.045867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:58.045834    2573 generic.go:356] "Generic (PLEG): container finished" podID="36b79698-3da3-4fa8-bb26-34481b34daaa" containerID="9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370" exitCode=137
Sep 10 21:29:58.046303 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:58.045877    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-a54104"
Sep 10 21:29:58.046303 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:58.045928    2573 scope.go:117] "RemoveContainer" containerID="9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370"
Sep 10 21:29:58.055008 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:58.054994    2573 scope.go:117] "RemoveContainer" containerID="9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370"
Sep 10 21:29:58.055257 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:58.055241    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370\": container with ID starting with 9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370 not found: ID does not exist" containerID="9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370"
Sep 10 21:29:58.055307 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:58.055264    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370"} err="failed to get container status \"9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370\": rpc error: code = NotFound desc = could not find container \"9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370\": container with ID starting with 9955144a84f161b76d89e72560edefa594f024471b42e603d0c5d0a54d4fc370 not found: ID does not exist"
Sep 10 21:29:58.055832 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:29:58.055808    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7791-a54104\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="36b79698-3da3-4fa8-bb26-34481b34daaa" pod="opendatahub/test-curl-7791-a54104"
Sep 10 21:29:58.680057 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:29:58.680028    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36b79698-3da3-4fa8-bb26-34481b34daaa" path="/var/lib/kubelet/pods/36b79698-3da3-4fa8-bb26-34481b34daaa/volumes"
Sep 10 21:30:00.143474 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.143439    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g"]
Sep 10 21:30:00.147103 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.147078    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs"]
Sep 10 21:30:00.147237 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.147183    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g"
Sep 10 21:30:00.149134 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.149113    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:30:00.149635 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.149618    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs"
Sep 10 21:30:00.157221 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.157199    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g"]
Sep 10 21:30:00.160672 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.160649    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g"
Sep 10 21:30:00.166051 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.166030    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk"]
Sep 10 21:30:00.167635 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.167619    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs"
Sep 10 21:30:00.169321 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.169109    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk"
Sep 10 21:30:00.179673 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.179650    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs"]
Sep 10 21:30:00.180560 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.180280    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk"]
Sep 10 21:30:00.216036 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.215989    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk"
Sep 10 21:30:00.451464 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.451432    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g"]
Sep 10 21:30:00.453422 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:30:00.453394    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6808a55b_8d58_43fa_a2ea_70428653e272.slice/crio-dc9103bd96741420c7e7680a43f09322de98cba84b9ff32da10d490ba65a1239 WatchSource:0}: Error finding container dc9103bd96741420c7e7680a43f09322de98cba84b9ff32da10d490ba65a1239: Status 404 returned error can't find the container with id dc9103bd96741420c7e7680a43f09322de98cba84b9ff32da10d490ba65a1239
Sep 10 21:30:00.454305 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.454286    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs"]
Sep 10 21:30:00.454743 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:30:00.454720    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod870622d9_6221_4b78_a5a3_15ea1f941fc8.slice/crio-2d8991789ea982f5cf30bc01eb2d46d5e9dcd3ec553575d5ceef1f7d00f9f29d WatchSource:0}: Error finding container 2d8991789ea982f5cf30bc01eb2d46d5e9dcd3ec553575d5ceef1f7d00f9f29d: Status 404 returned error can't find the container with id 2d8991789ea982f5cf30bc01eb2d46d5e9dcd3ec553575d5ceef1f7d00f9f29d
Sep 10 21:30:00.505153 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:00.505130    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk"]
Sep 10 21:30:00.506642 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:30:00.506617    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcfcd359_3cef_47a0_986c_552d7af095f5.slice/crio-ff14b6261e824462fd143932d23db58bb66287947fadcaa32ec86b98f04560fd WatchSource:0}: Error finding container ff14b6261e824462fd143932d23db58bb66287947fadcaa32ec86b98f04560fd: Status 404 returned error can't find the container with id ff14b6261e824462fd143932d23db58bb66287947fadcaa32ec86b98f04560fd
Sep 10 21:30:01.064927 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:01.064817    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g" event={"ID":"6808a55b-8d58-43fa-a2ea-70428653e272","Type":"ContainerStarted","Data":"97e001997617a2b913af507142966851ae472c0ea7a2061b55650d92ad045517"}
Sep 10 21:30:01.064927 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:01.064873    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g" event={"ID":"6808a55b-8d58-43fa-a2ea-70428653e272","Type":"ContainerStarted","Data":"dc9103bd96741420c7e7680a43f09322de98cba84b9ff32da10d490ba65a1239"}
Sep 10 21:30:01.066527 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:01.066490    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk" event={"ID":"bcfcd359-3cef-47a0-986c-552d7af095f5","Type":"ContainerStarted","Data":"bbf04d1541699d482a3245258ddb8990cbdcf94375ea1979b9608275d0cfff5a"}
Sep 10 21:30:01.066677 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:01.066541    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk" event={"ID":"bcfcd359-3cef-47a0-986c-552d7af095f5","Type":"ContainerStarted","Data":"ff14b6261e824462fd143932d23db58bb66287947fadcaa32ec86b98f04560fd"}
Sep 10 21:30:01.068061 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:01.068032    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs" event={"ID":"870622d9-6221-4b78-a5a3-15ea1f941fc8","Type":"ContainerStarted","Data":"99e21933839eb450b5423f73d11be5367d8f98e117b4dc04d6a61c4d0d541405"}
Sep 10 21:30:01.068160 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:01.068089    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs" event={"ID":"870622d9-6221-4b78-a5a3-15ea1f941fc8","Type":"ContainerStarted","Data":"2d8991789ea982f5cf30bc01eb2d46d5e9dcd3ec553575d5ceef1f7d00f9f29d"}
Sep 10 21:30:01.081626 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:01.081574    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g" podStartSLOduration=1.081559109 podStartE2EDuration="1.081559109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:01.078986593 +0000 UTC m=+1177.012653405" watchObservedRunningTime="2026-09-10 21:30:01.081559109 +0000 UTC m=+1177.015225921"
Sep 10 21:30:01.094197 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:01.094156    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs" podStartSLOduration=1.094142962 podStartE2EDuration="1.094142962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:01.09222521 +0000 UTC m=+1177.025892025" watchObservedRunningTime="2026-09-10 21:30:01.094142962 +0000 UTC m=+1177.027809775"
Sep 10 21:30:01.106191 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:01.106151    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk" podStartSLOduration=1.10613827 podStartE2EDuration="1.10613827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:01.104347251 +0000 UTC m=+1177.038014066" watchObservedRunningTime="2026-09-10 21:30:01.10613827 +0000 UTC m=+1177.039805136"
Sep 10 21:30:02.074914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:02.074879    2573 generic.go:356] "Generic (PLEG): container finished" podID="6808a55b-8d58-43fa-a2ea-70428653e272" containerID="97e001997617a2b913af507142966851ae472c0ea7a2061b55650d92ad045517" exitCode=0
Sep 10 21:30:02.075306 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:02.074966    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g" event={"ID":"6808a55b-8d58-43fa-a2ea-70428653e272","Type":"ContainerDied","Data":"97e001997617a2b913af507142966851ae472c0ea7a2061b55650d92ad045517"}
Sep 10 21:30:02.076557 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:02.076533    2573 generic.go:356] "Generic (PLEG): container finished" podID="bcfcd359-3cef-47a0-986c-552d7af095f5" containerID="bbf04d1541699d482a3245258ddb8990cbdcf94375ea1979b9608275d0cfff5a" exitCode=0
Sep 10 21:30:02.076702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:02.076616    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk" event={"ID":"bcfcd359-3cef-47a0-986c-552d7af095f5","Type":"ContainerDied","Data":"bbf04d1541699d482a3245258ddb8990cbdcf94375ea1979b9608275d0cfff5a"}
Sep 10 21:30:02.078181 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:02.078161    2573 generic.go:356] "Generic (PLEG): container finished" podID="870622d9-6221-4b78-a5a3-15ea1f941fc8" containerID="99e21933839eb450b5423f73d11be5367d8f98e117b4dc04d6a61c4d0d541405" exitCode=0
Sep 10 21:30:02.078292 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:02.078202    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs" event={"ID":"870622d9-6221-4b78-a5a3-15ea1f941fc8","Type":"ContainerDied","Data":"99e21933839eb450b5423f73d11be5367d8f98e117b4dc04d6a61c4d0d541405"}
Sep 10 21:30:03.235040 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:03.235009    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g"
Sep 10 21:30:03.281211 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:03.281183    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk"
Sep 10 21:30:03.284344 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:03.284327    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs"
Sep 10 21:30:04.090712 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:04.090682    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs"
Sep 10 21:30:04.090928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:04.090718    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs" event={"ID":"870622d9-6221-4b78-a5a3-15ea1f941fc8","Type":"ContainerDied","Data":"2d8991789ea982f5cf30bc01eb2d46d5e9dcd3ec553575d5ceef1f7d00f9f29d"}
Sep 10 21:30:04.090928 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:04.090763    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d8991789ea982f5cf30bc01eb2d46d5e9dcd3ec553575d5ceef1f7d00f9f29d"
Sep 10 21:30:04.092103 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:04.092083    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g"
Sep 10 21:30:04.092259 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:04.092108    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817930-8x79g" event={"ID":"6808a55b-8d58-43fa-a2ea-70428653e272","Type":"ContainerDied","Data":"dc9103bd96741420c7e7680a43f09322de98cba84b9ff32da10d490ba65a1239"}
Sep 10 21:30:04.092259 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:04.092147    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc9103bd96741420c7e7680a43f09322de98cba84b9ff32da10d490ba65a1239"
Sep 10 21:30:04.093487 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:04.093463    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk" event={"ID":"bcfcd359-3cef-47a0-986c-552d7af095f5","Type":"ContainerDied","Data":"ff14b6261e824462fd143932d23db58bb66287947fadcaa32ec86b98f04560fd"}
Sep 10 21:30:04.093487 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:04.093484    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk"
Sep 10 21:30:04.093625 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:04.093503    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff14b6261e824462fd143932d23db58bb66287947fadcaa32ec86b98f04560fd"
Sep 10 21:30:07.728722 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:07.728689    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:30:12.520077 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.520043    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"]
Sep 10 21:30:12.520616 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.520595    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6808a55b-8d58-43fa-a2ea-70428653e272" containerName="cleanup"
Sep 10 21:30:12.520701 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.520619    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6808a55b-8d58-43fa-a2ea-70428653e272" containerName="cleanup"
Sep 10 21:30:12.520701 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.520640    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="870622d9-6221-4b78-a5a3-15ea1f941fc8" containerName="cleanup"
Sep 10 21:30:12.520701 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.520650    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="870622d9-6221-4b78-a5a3-15ea1f941fc8" containerName="cleanup"
Sep 10 21:30:12.520701 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.520662    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcfcd359-3cef-47a0-986c-552d7af095f5" containerName="cleanup"
Sep 10 21:30:12.520701 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.520670    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcfcd359-3cef-47a0-986c-552d7af095f5" containerName="cleanup"
Sep 10 21:30:12.520970 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.520805    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6808a55b-8d58-43fa-a2ea-70428653e272" containerName="cleanup"
Sep 10 21:30:12.520970 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.520823    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="870622d9-6221-4b78-a5a3-15ea1f941fc8" containerName="cleanup"
Sep 10 21:30:12.520970 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.520835    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcfcd359-3cef-47a0-986c-552d7af095f5" containerName="cleanup"
Sep 10 21:30:12.523149 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.523129    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.525460 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.525440    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"default-dockercfg-mf4gw\""
Sep 10 21:30:12.526019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.526002    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"kube-root-ca.crt\""
Sep 10 21:30:12.526132 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.526065    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"openshift-service-ca.crt\""
Sep 10 21:30:12.526203 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.526143    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"auth-test-model-1f3879-kserve-self-signed-certs\""
Sep 10 21:30:12.538975 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.538952    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"]
Sep 10 21:30:12.550144 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.550122    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-model-cache\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.550255 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.550152    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpkp5\" (UniqueName: \"kubernetes.io/projected/567256a5-37a1-40ab-8aa9-cfb06529c159-kube-api-access-lpkp5\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.550255 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.550197    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/567256a5-37a1-40ab-8aa9-cfb06529c159-tls-certs\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.550357 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.550334    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-home\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.550394 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.550376    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-tmp-dir\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.550440 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.550428    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-dshm\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.651235 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.651192    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-home\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.651235 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.651231    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-tmp-dir\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.651432 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.651255    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-dshm\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.651432 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.651302    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-model-cache\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.651432 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.651319    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpkp5\" (UniqueName: \"kubernetes.io/projected/567256a5-37a1-40ab-8aa9-cfb06529c159-kube-api-access-lpkp5\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.651432 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.651367    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/567256a5-37a1-40ab-8aa9-cfb06529c159-tls-certs\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.651669 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.651647    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-home\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.651730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.651687    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-model-cache\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.651730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.651702    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-tmp-dir\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.653743 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.653723    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-dshm\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.654011 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.653993    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/567256a5-37a1-40ab-8aa9-cfb06529c159-tls-certs\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.662912 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.662885    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpkp5\" (UniqueName: \"kubernetes.io/projected/567256a5-37a1-40ab-8aa9-cfb06529c159-kube-api-access-lpkp5\") pod \"auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:12.832518 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:12.832484    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:13.136633 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:13.136597    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"]
Sep 10 21:30:13.137718 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:30:13.137694    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod567256a5_37a1_40ab_8aa9_cfb06529c159.slice/crio-51592a63ab0998ee9f7c2819539cde5ad84cf4fb661f8343b7fbae3ce6cc438c WatchSource:0}: Error finding container 51592a63ab0998ee9f7c2819539cde5ad84cf4fb661f8343b7fbae3ce6cc438c: Status 404 returned error can't find the container with id 51592a63ab0998ee9f7c2819539cde5ad84cf4fb661f8343b7fbae3ce6cc438c
Sep 10 21:30:13.145492 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:13.145468    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4" event={"ID":"567256a5-37a1-40ab-8aa9-cfb06529c159","Type":"ContainerStarted","Data":"51592a63ab0998ee9f7c2819539cde5ad84cf4fb661f8343b7fbae3ce6cc438c"}
Sep 10 21:30:14.152837 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:14.152802    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4" event={"ID":"567256a5-37a1-40ab-8aa9-cfb06529c159","Type":"ContainerStarted","Data":"dfb0817e5398bcd6dd487fba2c766562d1dcb7bd315609d6bd6d99e7897b930c"}
Sep 10 21:30:14.170683 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:14.170632    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4" podStartSLOduration=2.170619078 podStartE2EDuration="2.170619078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:14.169496931 +0000 UTC m=+1190.103163744" watchObservedRunningTime="2026-09-10 21:30:14.170619078 +0000 UTC m=+1190.104285886"
Sep 10 21:30:18.181161 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:18.181131    2573 generic.go:356] "Generic (PLEG): container finished" podID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerID="9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da" exitCode=0
Sep 10 21:30:18.181581 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:18.181201    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" event={"ID":"e4ddc4d1-72bc-48f4-974a-6311231611e1","Type":"ContainerDied","Data":"9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da"}
Sep 10 21:30:18.181630 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:18.181600    2573 scope.go:117] "RemoveContainer" containerID="9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da"
Sep 10 21:30:18.182695 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:18.182666    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:30:18.182824 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:18.182752    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:30:18.182899 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:18.182823    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:30:19.193573 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:19.193526    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" event={"ID":"e4ddc4d1-72bc-48f4-974a-6311231611e1","Type":"ContainerStarted","Data":"da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26"}
Sep 10 21:30:19.765696 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:19.765658    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:30:19.765919 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:19.765894    2573 patch_prober.go:28] interesting pod/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 21:30:19.766002 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:19.765951    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 21:30:19.916713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:19.916675    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:30:20.765892 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:20.765859    2573 patch_prober.go:28] interesting pod/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 21:30:20.766292 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:20.765928    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 21:30:21.765954 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:21.765917    2573 patch_prober.go:28] interesting pod/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 21:30:21.766326 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:21.765979    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 21:30:22.765967 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:22.765930    2573 patch_prober.go:28] interesting pod/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 21:30:22.766373 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:22.765990    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 21:30:22.833500 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:22.833460    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:22.833680 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:22.833530    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:22.845795 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:22.845757    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:23.230278 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:23.230240    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:30:23.765641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:23.765609    2573 patch_prober.go:28] interesting pod/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 21:30:23.765831 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:23.765660    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 21:30:24.565601 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:24.565565    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:30:24.566214 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:24.566198    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:30:24.669457 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:24.669415    2573 scope.go:117] "RemoveContainer" containerID="64db6ace0722856d32e664cdb6db63e8f21c45cd4f3563426faf52c63e86d9fc"
Sep 10 21:30:24.784890 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:24.784841    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:30:24.785449 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:24.785397    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:30:24.786262 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:24.786241    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:30:50.312948 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.312915    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"]
Sep 10 21:30:50.315555 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.315533    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.318481 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.318447    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7ce89dd6\"/\"kube-root-ca.crt\""
Sep 10 21:30:50.318481 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.318460    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7ce89dd6\"/\"e2e-lifecycle-model-7ce89dd6-kserve-self-signed-certs\""
Sep 10 21:30:50.318659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.318459    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7ce89dd6\"/\"openshift-service-ca.crt\""
Sep 10 21:30:50.318659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.318544    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7ce89dd6\"/\"default-dockercfg-bw644\""
Sep 10 21:30:50.326598 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.326576    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"]
Sep 10 21:30:50.430935 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.430904    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-dshm\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.431089 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.430942    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d437c53f-2cd1-4967-a525-d4c2c73e9545-tls-certs\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.431089 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.430965    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlk7w\" (UniqueName: \"kubernetes.io/projected/d437c53f-2cd1-4967-a525-d4c2c73e9545-kube-api-access-jlk7w\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.431089 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.431033    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-tmp-dir\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.431089 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.431083    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-model-cache\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.431241 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.431105    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-home\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.531940 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.531896    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlk7w\" (UniqueName: \"kubernetes.io/projected/d437c53f-2cd1-4967-a525-d4c2c73e9545-kube-api-access-jlk7w\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.532329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.531960    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-tmp-dir\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.532329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.532008    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-model-cache\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.532329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.532039    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-home\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.532329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.532116    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-dshm\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.532329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.532163    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d437c53f-2cd1-4967-a525-d4c2c73e9545-tls-certs\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.532760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.532475    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-tmp-dir\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.532760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.532502    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-model-cache\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.532760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.532616    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-home\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.534461 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.534435    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-dshm\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.534670 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.534653    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d437c53f-2cd1-4967-a525-d4c2c73e9545-tls-certs\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.540267 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.540246    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlk7w\" (UniqueName: \"kubernetes.io/projected/d437c53f-2cd1-4967-a525-d4c2c73e9545-kube-api-access-jlk7w\") pod \"e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") " pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.626157 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.626128    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:30:50.898457 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:50.898427    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"]
Sep 10 21:30:50.900160 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:30:50.900131    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd437c53f_2cd1_4967_a525_d4c2c73e9545.slice/crio-7ce987b4dee3df8a222548a3044acd2a633a6e22e39062c3e071b5e814654a45 WatchSource:0}: Error finding container 7ce987b4dee3df8a222548a3044acd2a633a6e22e39062c3e071b5e814654a45: Status 404 returned error can't find the container with id 7ce987b4dee3df8a222548a3044acd2a633a6e22e39062c3e071b5e814654a45
Sep 10 21:30:51.385236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:51.385197    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf" event={"ID":"d437c53f-2cd1-4967-a525-d4c2c73e9545","Type":"ContainerStarted","Data":"28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597"}
Sep 10 21:30:51.385236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:51.385243    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf" event={"ID":"d437c53f-2cd1-4967-a525-d4c2c73e9545","Type":"ContainerStarted","Data":"7ce987b4dee3df8a222548a3044acd2a633a6e22e39062c3e071b5e814654a45"}
Sep 10 21:30:51.404582 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:51.404528    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf" podStartSLOduration=1.404512526 podStartE2EDuration="1.404512526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:30:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:51.402122831 +0000 UTC m=+1227.335789647" watchObservedRunningTime="2026-09-10 21:30:51.404512526 +0000 UTC m=+1227.338179341"
Sep 10 21:30:51.415588 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:30:51.415558    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:31:00.626761 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:00.626723    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:31:00.627281 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:00.626825    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:31:00.639495 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:00.639464    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:31:01.460091 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:01.460063    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:31:09.918429 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:09.918386    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:31:24.717349 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:24.717268    2573 scope.go:117] "RemoveContainer" containerID="7cedae263f6e98617c88f7d4f112facedcbc35d9ed416ef017a8ac702e51c085"
Sep 10 21:31:24.726300 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:24.726277    2573 scope.go:117] "RemoveContainer" containerID="63e06bd4e5564f3944b59297fac3863556756e2b7c5dde402d4617288efa5cb1"
Sep 10 21:31:25.213890 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:25.213854    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:31:41.426817 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:41.426767    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"]
Sep 10 21:31:43.656372 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:43.656326    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:31:56.446048 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.445999    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf" podUID="d437c53f-2cd1-4967-a525-d4c2c73e9545" containerName="main" containerID="cri-o://28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597" gracePeriod=45
Sep 10 21:31:56.696904 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.696838    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:31:56.774284 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.774254    2573 generic.go:356] "Generic (PLEG): container finished" podID="d437c53f-2cd1-4967-a525-d4c2c73e9545" containerID="28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597" exitCode=0
Sep 10 21:31:56.774438 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.774303    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"
Sep 10 21:31:56.774438 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.774331    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf" event={"ID":"d437c53f-2cd1-4967-a525-d4c2c73e9545","Type":"ContainerDied","Data":"28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597"}
Sep 10 21:31:56.774438 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.774375    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf" event={"ID":"d437c53f-2cd1-4967-a525-d4c2c73e9545","Type":"ContainerDied","Data":"7ce987b4dee3df8a222548a3044acd2a633a6e22e39062c3e071b5e814654a45"}
Sep 10 21:31:56.774438 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.774393    2573 scope.go:117] "RemoveContainer" containerID="28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597"
Sep 10 21:31:56.783295 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.783280    2573 scope.go:117] "RemoveContainer" containerID="28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597"
Sep 10 21:31:56.783572 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:31:56.783542    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597\": container with ID starting with 28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597 not found: ID does not exist" containerID="28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597"
Sep 10 21:31:56.783668 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.783571    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597"} err="failed to get container status \"28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597\": rpc error: code = NotFound desc = could not find container \"28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597\": container with ID starting with 28a2cbe6cd9064f14796bd46000b8a1d047fb7fe830ea6f8d603e0ab42c11597 not found: ID does not exist"
Sep 10 21:31:56.795339 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.795319    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-dshm\") pod \"d437c53f-2cd1-4967-a525-d4c2c73e9545\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") "
Sep 10 21:31:56.795437 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.795400    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-model-cache\") pod \"d437c53f-2cd1-4967-a525-d4c2c73e9545\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") "
Sep 10 21:31:56.795487 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.795433    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlk7w\" (UniqueName: \"kubernetes.io/projected/d437c53f-2cd1-4967-a525-d4c2c73e9545-kube-api-access-jlk7w\") pod \"d437c53f-2cd1-4967-a525-d4c2c73e9545\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") "
Sep 10 21:31:56.795487 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.795465    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-home\") pod \"d437c53f-2cd1-4967-a525-d4c2c73e9545\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") "
Sep 10 21:31:56.795595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.795500    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-tmp-dir\") pod \"d437c53f-2cd1-4967-a525-d4c2c73e9545\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") "
Sep 10 21:31:56.795595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.795591    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d437c53f-2cd1-4967-a525-d4c2c73e9545-tls-certs\") pod \"d437c53f-2cd1-4967-a525-d4c2c73e9545\" (UID: \"d437c53f-2cd1-4967-a525-d4c2c73e9545\") "
Sep 10 21:31:56.795732 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.795675    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-model-cache" (OuterVolumeSpecName: "model-cache") pod "d437c53f-2cd1-4967-a525-d4c2c73e9545" (UID: "d437c53f-2cd1-4967-a525-d4c2c73e9545"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:31:56.795810 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.795751    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-home" (OuterVolumeSpecName: "home") pod "d437c53f-2cd1-4967-a525-d4c2c73e9545" (UID: "d437c53f-2cd1-4967-a525-d4c2c73e9545"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:31:56.795882 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.795863    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d437c53f-2cd1-4967-a525-d4c2c73e9545" (UID: "d437c53f-2cd1-4967-a525-d4c2c73e9545"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:31:56.796107 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.796086    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:31:56.796203 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.796113    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:31:56.796203 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.796127    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:31:56.797637 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.797611    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-dshm" (OuterVolumeSpecName: "dshm") pod "d437c53f-2cd1-4967-a525-d4c2c73e9545" (UID: "d437c53f-2cd1-4967-a525-d4c2c73e9545"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:31:56.797753 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.797662    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d437c53f-2cd1-4967-a525-d4c2c73e9545-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d437c53f-2cd1-4967-a525-d4c2c73e9545" (UID: "d437c53f-2cd1-4967-a525-d4c2c73e9545"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:31:56.797753 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.797701    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d437c53f-2cd1-4967-a525-d4c2c73e9545-kube-api-access-jlk7w" (OuterVolumeSpecName: "kube-api-access-jlk7w") pod "d437c53f-2cd1-4967-a525-d4c2c73e9545" (UID: "d437c53f-2cd1-4967-a525-d4c2c73e9545"). InnerVolumeSpecName "kube-api-access-jlk7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:31:56.896979 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.896949    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlk7w\" (UniqueName: \"kubernetes.io/projected/d437c53f-2cd1-4967-a525-d4c2c73e9545-kube-api-access-jlk7w\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:31:56.896979 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.896976    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d437c53f-2cd1-4967-a525-d4c2c73e9545-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:31:56.896979 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:56.896987    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d437c53f-2cd1-4967-a525-d4c2c73e9545-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:31:57.098942 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:57.098910    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"]
Sep 10 21:31:57.102185 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:57.102159    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7ce89dd6/e2e-lifecycle-model-7ce89dd6-kserve-565dcd7b75-fkgcf"]
Sep 10 21:31:58.679840 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:31:58.679806    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d437c53f-2cd1-4967-a525-d4c2c73e9545" path="/var/lib/kubelet/pods/d437c53f-2cd1-4967-a525-d4c2c73e9545/volumes"
Sep 10 21:32:00.411765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:00.411729    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:32:11.213022 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:11.212990    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"]
Sep 10 21:32:11.213427 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:11.213223    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy" containerID="cri-o://da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26" gracePeriod=30
Sep 10 21:32:16.020456 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.020420    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:32:16.472650 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.472624    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:32:16.589226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589131    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-certs\") pod \"e4ddc4d1-72bc-48f4-974a-6311231611e1\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") "
Sep 10 21:32:16.589226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589180    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-data\") pod \"e4ddc4d1-72bc-48f4-974a-6311231611e1\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") "
Sep 10 21:32:16.589226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589224    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t22kp\" (UniqueName: \"kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-kube-api-access-t22kp\") pod \"e4ddc4d1-72bc-48f4-974a-6311231611e1\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") "
Sep 10 21:32:16.589517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589251    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-envoy\") pod \"e4ddc4d1-72bc-48f4-974a-6311231611e1\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") "
Sep 10 21:32:16.589517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589276    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-socket\") pod \"e4ddc4d1-72bc-48f4-974a-6311231611e1\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") "
Sep 10 21:32:16.589517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589302    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-token\") pod \"e4ddc4d1-72bc-48f4-974a-6311231611e1\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") "
Sep 10 21:32:16.589517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589376    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ddc4d1-72bc-48f4-974a-6311231611e1-istiod-ca-cert\") pod \"e4ddc4d1-72bc-48f4-974a-6311231611e1\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") "
Sep 10 21:32:16.589517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589413    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-podinfo\") pod \"e4ddc4d1-72bc-48f4-974a-6311231611e1\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") "
Sep 10 21:32:16.589517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589464    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-credential-socket\") pod \"e4ddc4d1-72bc-48f4-974a-6311231611e1\" (UID: \"e4ddc4d1-72bc-48f4-974a-6311231611e1\") "
Sep 10 21:32:16.589517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589476    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4ddc4d1-72bc-48f4-974a-6311231611e1" (UID: "e4ddc4d1-72bc-48f4-974a-6311231611e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:16.589911 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589537    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4ddc4d1-72bc-48f4-974a-6311231611e1" (UID: "e4ddc4d1-72bc-48f4-974a-6311231611e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:16.589911 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589643    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4ddc4d1-72bc-48f4-974a-6311231611e1" (UID: "e4ddc4d1-72bc-48f4-974a-6311231611e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:16.589911 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589830    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ddc4d1-72bc-48f4-974a-6311231611e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4ddc4d1-72bc-48f4-974a-6311231611e1" (UID: "e4ddc4d1-72bc-48f4-974a-6311231611e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:32:16.589911 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589908    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:16.590101 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589927    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:16.590101 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589937    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4ddc4d1-72bc-48f4-974a-6311231611e1" (UID: "e4ddc4d1-72bc-48f4-974a-6311231611e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:16.590101 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.589946    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:16.591701 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.591666    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-kube-api-access-t22kp" (OuterVolumeSpecName: "kube-api-access-t22kp") pod "e4ddc4d1-72bc-48f4-974a-6311231611e1" (UID: "e4ddc4d1-72bc-48f4-974a-6311231611e1"). InnerVolumeSpecName "kube-api-access-t22kp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:16.591819 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.591689    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4ddc4d1-72bc-48f4-974a-6311231611e1" (UID: "e4ddc4d1-72bc-48f4-974a-6311231611e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:16.591967 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.591942    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4ddc4d1-72bc-48f4-974a-6311231611e1" (UID: "e4ddc4d1-72bc-48f4-974a-6311231611e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:16.592047 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.592030    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4ddc4d1-72bc-48f4-974a-6311231611e1" (UID: "e4ddc4d1-72bc-48f4-974a-6311231611e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:32:16.690592 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.690563    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:16.690592 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.690588    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ddc4d1-72bc-48f4-974a-6311231611e1-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:16.690747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.690598    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:16.690747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.690606    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:16.690747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.690615    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t22kp\" (UniqueName: \"kubernetes.io/projected/e4ddc4d1-72bc-48f4-974a-6311231611e1-kube-api-access-t22kp\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:16.690747 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.690625    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ddc4d1-72bc-48f4-974a-6311231611e1-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:16.894329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.894233    2573 generic.go:356] "Generic (PLEG): container finished" podID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerID="da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26" exitCode=0
Sep 10 21:32:16.894329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.894283    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"
Sep 10 21:32:16.894329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.894310    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" event={"ID":"e4ddc4d1-72bc-48f4-974a-6311231611e1","Type":"ContainerDied","Data":"da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26"}
Sep 10 21:32:16.894583 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.894371    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc" event={"ID":"e4ddc4d1-72bc-48f4-974a-6311231611e1","Type":"ContainerDied","Data":"4a590dcca846955513673f836f29607356289ea1bf3f2174b3d48811e8bb7d0b"}
Sep 10 21:32:16.894583 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.894404    2573 scope.go:117] "RemoveContainer" containerID="da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26"
Sep 10 21:32:16.904827 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.904807    2573 scope.go:117] "RemoveContainer" containerID="9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da"
Sep 10 21:32:16.912646 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.912618    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"]
Sep 10 21:32:16.916374 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.916356    2573 scope.go:117] "RemoveContainer" containerID="da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26"
Sep 10 21:32:16.916750 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:32:16.916726    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26\": container with ID starting with da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26 not found: ID does not exist" containerID="da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26"
Sep 10 21:32:16.916867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.916758    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26"} err="failed to get container status \"da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26\": rpc error: code = NotFound desc = could not find container \"da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26\": container with ID starting with da0c1e3d81488e6a424ea50dde11f8b172d99d9a526b7fd881ffff78a62f9c26 not found: ID does not exist"
Sep 10 21:32:16.916867 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.916803    2573 scope.go:117] "RemoveContainer" containerID="9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da"
Sep 10 21:32:16.917175 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:32:16.917145    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da\": container with ID starting with 9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da not found: ID does not exist" containerID="9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da"
Sep 10 21:32:16.917288 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.917181    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da"} err="failed to get container status \"9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da\": rpc error: code = NotFound desc = could not find container \"9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da\": container with ID starting with 9791b554549a664083e58117837e74499cba52b5d1d8553418b0bc030da979da not found: ID does not exist"
Sep 10 21:32:16.919940 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:16.919920    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7ce89dd6-openshift-default-558d75f55b-58xzc"]
Sep 10 21:32:18.679850 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:18.679813    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" path="/var/lib/kubelet/pods/e4ddc4d1-72bc-48f4-974a-6311231611e1/volumes"
Sep 10 21:32:24.753402 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:24.753373    2573 scope.go:117] "RemoveContainer" containerID="b4c3c0af66ee1f5a0daba1bef6337be74101a4a231e77019cc3ae7ef68645d7d"
Sep 10 21:32:28.428083 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.428046    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7794-5fd6fe"]
Sep 10 21:32:28.428694 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.428599    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy"
Sep 10 21:32:28.428694 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.428619    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy"
Sep 10 21:32:28.428694 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.428635    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d437c53f-2cd1-4967-a525-d4c2c73e9545" containerName="main"
Sep 10 21:32:28.428694 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.428644    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d437c53f-2cd1-4967-a525-d4c2c73e9545" containerName="main"
Sep 10 21:32:28.428694 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.428662    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy"
Sep 10 21:32:28.428694 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.428672    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy"
Sep 10 21:32:28.429047 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.428802    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d437c53f-2cd1-4967-a525-d4c2c73e9545" containerName="main"
Sep 10 21:32:28.429047 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.428814    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy"
Sep 10 21:32:28.429047 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.428824    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ddc4d1-72bc-48f4-974a-6311231611e1" containerName="istio-proxy"
Sep 10 21:32:28.431771 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.431751    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-5fd6fe"
Sep 10 21:32:28.434350 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.434331    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dxcl2\""
Sep 10 21:32:28.441169 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.441145    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7794-5fd6fe"]
Sep 10 21:32:28.606259 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.606216    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g9xg\" (UniqueName: \"kubernetes.io/projected/10482e6a-230a-493d-876d-16c5bf7b5c2c-kube-api-access-6g9xg\") pod \"test-curl-7794-5fd6fe\" (UID: \"10482e6a-230a-493d-876d-16c5bf7b5c2c\") " pod="opendatahub/test-curl-7794-5fd6fe"
Sep 10 21:32:28.707879 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.707762    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g9xg\" (UniqueName: \"kubernetes.io/projected/10482e6a-230a-493d-876d-16c5bf7b5c2c-kube-api-access-6g9xg\") pod \"test-curl-7794-5fd6fe\" (UID: \"10482e6a-230a-493d-876d-16c5bf7b5c2c\") " pod="opendatahub/test-curl-7794-5fd6fe"
Sep 10 21:32:28.717052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.717028    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g9xg\" (UniqueName: \"kubernetes.io/projected/10482e6a-230a-493d-876d-16c5bf7b5c2c-kube-api-access-6g9xg\") pod \"test-curl-7794-5fd6fe\" (UID: \"10482e6a-230a-493d-876d-16c5bf7b5c2c\") " pod="opendatahub/test-curl-7794-5fd6fe"
Sep 10 21:32:28.741677 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:28.741648    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-5fd6fe"
Sep 10 21:32:29.012917 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:29.012888    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7794-5fd6fe"]
Sep 10 21:32:29.014449 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:32:29.014412    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10482e6a_230a_493d_876d_16c5bf7b5c2c.slice/crio-68150059bf1cd94c45de2b90b60da6bb82c953f6a6d55c8bc47e7e2347dede21 WatchSource:0}: Error finding container 68150059bf1cd94c45de2b90b60da6bb82c953f6a6d55c8bc47e7e2347dede21: Status 404 returned error can't find the container with id 68150059bf1cd94c45de2b90b60da6bb82c953f6a6d55c8bc47e7e2347dede21
Sep 10 21:32:29.970326 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:29.970291    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7794-5fd6fe" event={"ID":"10482e6a-230a-493d-876d-16c5bf7b5c2c","Type":"ContainerStarted","Data":"8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85"}
Sep 10 21:32:29.970326 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:29.970332    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7794-5fd6fe" event={"ID":"10482e6a-230a-493d-876d-16c5bf7b5c2c","Type":"ContainerStarted","Data":"68150059bf1cd94c45de2b90b60da6bb82c953f6a6d55c8bc47e7e2347dede21"}
Sep 10 21:32:29.985845 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:29.985799    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7794-5fd6fe" podStartSLOduration=1.9857678600000002 podStartE2EDuration="1.98576786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:29.984378073 +0000 UTC m=+1325.918044885" watchObservedRunningTime="2026-09-10 21:32:29.98576786 +0000 UTC m=+1325.919434674"
Sep 10 21:32:30.571126 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:30.571069    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7794-5fd6fe"]
Sep 10 21:32:30.574708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:30.574679    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7794-5fd6fe"]
Sep 10 21:32:30.975793 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:32:30.975753    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7794-5fd6fe" container="test-curl-7794-5fd6fe"
Sep 10 21:32:31.418070 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:31.418030    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:32:31.907912 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:31.907877    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"]
Sep 10 21:32:31.981520 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:31.981482    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7794-5fd6fe" podUID="10482e6a-230a-493d-876d-16c5bf7b5c2c" containerName="test-curl-7794-5fd6fe" containerID="cri-o://8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85" gracePeriod=2
Sep 10 21:32:32.903460 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:32.903426    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"]
Sep 10 21:32:34.143329 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:34.143304    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-5fd6fe"
Sep 10 21:32:34.264653 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:34.264572    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g9xg\" (UniqueName: \"kubernetes.io/projected/10482e6a-230a-493d-876d-16c5bf7b5c2c-kube-api-access-6g9xg\") pod \"10482e6a-230a-493d-876d-16c5bf7b5c2c\" (UID: \"10482e6a-230a-493d-876d-16c5bf7b5c2c\") "
Sep 10 21:32:34.266694 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:34.266672    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10482e6a-230a-493d-876d-16c5bf7b5c2c-kube-api-access-6g9xg" (OuterVolumeSpecName: "kube-api-access-6g9xg") pod "10482e6a-230a-493d-876d-16c5bf7b5c2c" (UID: "10482e6a-230a-493d-876d-16c5bf7b5c2c"). InnerVolumeSpecName "kube-api-access-6g9xg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:34.365555 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:34.365518    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g9xg\" (UniqueName: \"kubernetes.io/projected/10482e6a-230a-493d-876d-16c5bf7b5c2c-kube-api-access-6g9xg\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:34.685642 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:34.685610    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10482e6a-230a-493d-876d-16c5bf7b5c2c" path="/var/lib/kubelet/pods/10482e6a-230a-493d-876d-16c5bf7b5c2c/volumes"
Sep 10 21:32:34.999794 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:34.999675    2573 generic.go:356] "Generic (PLEG): container finished" podID="10482e6a-230a-493d-876d-16c5bf7b5c2c" containerID="8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85" exitCode=137
Sep 10 21:32:34.999794 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:34.999722    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-5fd6fe"
Sep 10 21:32:34.999794 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:34.999787    2573 scope.go:117] "RemoveContainer" containerID="8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85"
Sep 10 21:32:35.008603 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:35.008576    2573 scope.go:117] "RemoveContainer" containerID="8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85"
Sep 10 21:32:35.008907 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:32:35.008887    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85\": container with ID starting with 8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85 not found: ID does not exist" containerID="8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85"
Sep 10 21:32:35.008993 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:35.008915    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85"} err="failed to get container status \"8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85\": rpc error: code = NotFound desc = could not find container \"8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85\": container with ID starting with 8071107eeb1acd22ad537c29c4c5fc2100176acb9e6fe237d99981cacbc46a85 not found: ID does not exist"
Sep 10 21:32:41.611123 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:41.611089    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:32:43.713921 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.713887    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"]
Sep 10 21:32:43.714292 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.714279    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10482e6a-230a-493d-876d-16c5bf7b5c2c" containerName="test-curl-7794-5fd6fe"
Sep 10 21:32:43.714292 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.714291    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="10482e6a-230a-493d-876d-16c5bf7b5c2c" containerName="test-curl-7794-5fd6fe"
Sep 10 21:32:43.714377 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.714367    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="10482e6a-230a-493d-876d-16c5bf7b5c2c" containerName="test-curl-7794-5fd6fe"
Sep 10 21:32:43.717556 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.717534    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.719532 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.719506    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-36b938\"/\"sub-test-model-36b938-kserve-self-signed-certs\""
Sep 10 21:32:43.726442 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.726419    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"]
Sep 10 21:32:43.761261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.761227    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89bfl\" (UniqueName: \"kubernetes.io/projected/2ef9e793-69e7-459d-a45c-320f73717f23-kube-api-access-89bfl\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.761425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.761266    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ef9e793-69e7-459d-a45c-320f73717f23-tls-certs\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.761425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.761305    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-dshm\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.761425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.761324    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-model-cache\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.761425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.761397    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-home\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.761565 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.761439    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-tmp-dir\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.862759 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.862724    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89bfl\" (UniqueName: \"kubernetes.io/projected/2ef9e793-69e7-459d-a45c-320f73717f23-kube-api-access-89bfl\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.862759 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.862763    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ef9e793-69e7-459d-a45c-320f73717f23-tls-certs\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.863018 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.862840    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-dshm\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.863018 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.862859    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-model-cache\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.863018 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.862928    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-home\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.863018 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.862963    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-tmp-dir\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.863366 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.863339    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-tmp-dir\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.863485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.863344    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-model-cache\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.863485 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.863397    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-home\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.865284 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.865258    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-dshm\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.865459 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.865438    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ef9e793-69e7-459d-a45c-320f73717f23-tls-certs\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:43.870829 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:43.870805    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89bfl\" (UniqueName: \"kubernetes.io/projected/2ef9e793-69e7-459d-a45c-320f73717f23-kube-api-access-89bfl\") pod \"sub-test-model-36b938-kserve-897946849-z7v2t\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") " pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:44.026880 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:44.026756    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:44.300532 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:44.300344    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"]
Sep 10 21:32:44.303482 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:32:44.303448    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ef9e793_69e7_459d_a45c_320f73717f23.slice/crio-2f3e33882ba2c10731cf31ab684231306f95703d7595f6f88a128e48c21bcb0b WatchSource:0}: Error finding container 2f3e33882ba2c10731cf31ab684231306f95703d7595f6f88a128e48c21bcb0b: Status 404 returned error can't find the container with id 2f3e33882ba2c10731cf31ab684231306f95703d7595f6f88a128e48c21bcb0b
Sep 10 21:32:45.068132 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:45.068097    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t" event={"ID":"2ef9e793-69e7-459d-a45c-320f73717f23","Type":"ContainerStarted","Data":"47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b"}
Sep 10 21:32:45.068674 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:45.068140    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t" event={"ID":"2ef9e793-69e7-459d-a45c-320f73717f23","Type":"ContainerStarted","Data":"2f3e33882ba2c10731cf31ab684231306f95703d7595f6f88a128e48c21bcb0b"}
Sep 10 21:32:45.091690 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:45.091583    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t" podStartSLOduration=2.091564946 podStartE2EDuration="2.091564946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:45.083892376 +0000 UTC m=+1341.017559204" watchObservedRunningTime="2026-09-10 21:32:45.091564946 +0000 UTC m=+1341.025231763"
Sep 10 21:32:46.927534 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:46.927454    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg" podUID="7a681998-2116-437f-831a-e26a564490d7" containerName="main" containerID="cri-o://bec9fd65b39d0fb212d2a0ff950e3a6357e1445b2afd597266ec5dfd731c1720" gracePeriod=45
Sep 10 21:32:46.960271 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:46.960239    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg" podUID="7a681998-2116-437f-831a-e26a564490d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": dial tcp 10.134.0.67:8000: connect: connection refused"
Sep 10 21:32:47.082302 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.082271    2573 generic.go:356] "Generic (PLEG): container finished" podID="7a681998-2116-437f-831a-e26a564490d7" containerID="bec9fd65b39d0fb212d2a0ff950e3a6357e1445b2afd597266ec5dfd731c1720" exitCode=0
Sep 10 21:32:47.082483 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.082346    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg" event={"ID":"7a681998-2116-437f-831a-e26a564490d7","Type":"ContainerDied","Data":"bec9fd65b39d0fb212d2a0ff950e3a6357e1445b2afd597266ec5dfd731c1720"}
Sep 10 21:32:47.176274 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.176248    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:32:47.293828 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.293711    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-model-cache\") pod \"7a681998-2116-437f-831a-e26a564490d7\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") "
Sep 10 21:32:47.293828 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.293798    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-tmp-dir\") pod \"7a681998-2116-437f-831a-e26a564490d7\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") "
Sep 10 21:32:47.294053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.293842    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24hk7\" (UniqueName: \"kubernetes.io/projected/7a681998-2116-437f-831a-e26a564490d7-kube-api-access-24hk7\") pod \"7a681998-2116-437f-831a-e26a564490d7\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") "
Sep 10 21:32:47.294053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.293879    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-home\") pod \"7a681998-2116-437f-831a-e26a564490d7\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") "
Sep 10 21:32:47.294053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.293911    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-dshm\") pod \"7a681998-2116-437f-831a-e26a564490d7\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") "
Sep 10 21:32:47.294053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.293975    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a681998-2116-437f-831a-e26a564490d7-tls-certs\") pod \"7a681998-2116-437f-831a-e26a564490d7\" (UID: \"7a681998-2116-437f-831a-e26a564490d7\") "
Sep 10 21:32:47.294245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.294043    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a681998-2116-437f-831a-e26a564490d7" (UID: "7a681998-2116-437f-831a-e26a564490d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:47.294245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.294058    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a681998-2116-437f-831a-e26a564490d7" (UID: "7a681998-2116-437f-831a-e26a564490d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:47.294245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.294158    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-home" (OuterVolumeSpecName: "home") pod "7a681998-2116-437f-831a-e26a564490d7" (UID: "7a681998-2116-437f-831a-e26a564490d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:47.294420 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.294383    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:47.294420 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.294400    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:47.294420 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.294409    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:47.296244 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.296214    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a681998-2116-437f-831a-e26a564490d7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a681998-2116-437f-831a-e26a564490d7" (UID: "7a681998-2116-437f-831a-e26a564490d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:32:47.296346 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.296241    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-dshm" (OuterVolumeSpecName: "dshm") pod "7a681998-2116-437f-831a-e26a564490d7" (UID: "7a681998-2116-437f-831a-e26a564490d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:47.296346 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.296304    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a681998-2116-437f-831a-e26a564490d7-kube-api-access-24hk7" (OuterVolumeSpecName: "kube-api-access-24hk7") pod "7a681998-2116-437f-831a-e26a564490d7" (UID: "7a681998-2116-437f-831a-e26a564490d7"). InnerVolumeSpecName "kube-api-access-24hk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:47.395713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.395673    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24hk7\" (UniqueName: \"kubernetes.io/projected/7a681998-2116-437f-831a-e26a564490d7-kube-api-access-24hk7\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:47.395713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.395709    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a681998-2116-437f-831a-e26a564490d7-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:47.395713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.395719    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a681998-2116-437f-831a-e26a564490d7-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:47.922740 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:47.922697    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4" podUID="567256a5-37a1-40ab-8aa9-cfb06529c159" containerName="main" containerID="cri-o://dfb0817e5398bcd6dd487fba2c766562d1dcb7bd315609d6bd6d99e7897b930c" gracePeriod=45
Sep 10 21:32:48.089392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.089355    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"
Sep 10 21:32:48.089920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.089389    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg" event={"ID":"7a681998-2116-437f-831a-e26a564490d7","Type":"ContainerDied","Data":"146a59aa781c3f6f0dc96a08cfa9230c9535d787d2e6ac81c8b1d4a91cec3f2c"}
Sep 10 21:32:48.089920 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.089456    2573 scope.go:117] "RemoveContainer" containerID="bec9fd65b39d0fb212d2a0ff950e3a6357e1445b2afd597266ec5dfd731c1720"
Sep 10 21:32:48.091959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.091922    2573 generic.go:356] "Generic (PLEG): container finished" podID="567256a5-37a1-40ab-8aa9-cfb06529c159" containerID="dfb0817e5398bcd6dd487fba2c766562d1dcb7bd315609d6bd6d99e7897b930c" exitCode=0
Sep 10 21:32:48.092059 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.091974    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4" event={"ID":"567256a5-37a1-40ab-8aa9-cfb06529c159","Type":"ContainerDied","Data":"dfb0817e5398bcd6dd487fba2c766562d1dcb7bd315609d6bd6d99e7897b930c"}
Sep 10 21:32:48.117000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.116973    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"]
Sep 10 21:32:48.119328 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.119303    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/auth-test-model-36b938-kserve-78ff7b7687-w8rlg"]
Sep 10 21:32:48.175834 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.175758    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:32:48.305115 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305081    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-model-cache\") pod \"567256a5-37a1-40ab-8aa9-cfb06529c159\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") "
Sep 10 21:32:48.305282 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305158    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpkp5\" (UniqueName: \"kubernetes.io/projected/567256a5-37a1-40ab-8aa9-cfb06529c159-kube-api-access-lpkp5\") pod \"567256a5-37a1-40ab-8aa9-cfb06529c159\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") "
Sep 10 21:32:48.305282 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305223    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/567256a5-37a1-40ab-8aa9-cfb06529c159-tls-certs\") pod \"567256a5-37a1-40ab-8aa9-cfb06529c159\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") "
Sep 10 21:32:48.305387 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305296    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-tmp-dir\") pod \"567256a5-37a1-40ab-8aa9-cfb06529c159\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") "
Sep 10 21:32:48.305387 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305320    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-dshm\") pod \"567256a5-37a1-40ab-8aa9-cfb06529c159\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") "
Sep 10 21:32:48.305387 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305359    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-home\") pod \"567256a5-37a1-40ab-8aa9-cfb06529c159\" (UID: \"567256a5-37a1-40ab-8aa9-cfb06529c159\") "
Sep 10 21:32:48.305387 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305361    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-model-cache" (OuterVolumeSpecName: "model-cache") pod "567256a5-37a1-40ab-8aa9-cfb06529c159" (UID: "567256a5-37a1-40ab-8aa9-cfb06529c159"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:48.305591 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305570    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "567256a5-37a1-40ab-8aa9-cfb06529c159" (UID: "567256a5-37a1-40ab-8aa9-cfb06529c159"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:48.305649 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305610    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-home" (OuterVolumeSpecName: "home") pod "567256a5-37a1-40ab-8aa9-cfb06529c159" (UID: "567256a5-37a1-40ab-8aa9-cfb06529c159"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:48.305871 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305849    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:48.305946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305877    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:48.305946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.305891    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:48.307484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.307460    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-dshm" (OuterVolumeSpecName: "dshm") pod "567256a5-37a1-40ab-8aa9-cfb06529c159" (UID: "567256a5-37a1-40ab-8aa9-cfb06529c159"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:48.307547 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.307509    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/567256a5-37a1-40ab-8aa9-cfb06529c159-kube-api-access-lpkp5" (OuterVolumeSpecName: "kube-api-access-lpkp5") pod "567256a5-37a1-40ab-8aa9-cfb06529c159" (UID: "567256a5-37a1-40ab-8aa9-cfb06529c159"). InnerVolumeSpecName "kube-api-access-lpkp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:48.307587 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.307558    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/567256a5-37a1-40ab-8aa9-cfb06529c159-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "567256a5-37a1-40ab-8aa9-cfb06529c159" (UID: "567256a5-37a1-40ab-8aa9-cfb06529c159"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:32:48.407261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.407220    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/567256a5-37a1-40ab-8aa9-cfb06529c159-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:48.407261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.407253    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpkp5\" (UniqueName: \"kubernetes.io/projected/567256a5-37a1-40ab-8aa9-cfb06529c159-kube-api-access-lpkp5\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:48.407261 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.407269    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/567256a5-37a1-40ab-8aa9-cfb06529c159-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:32:48.678853 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:48.678821    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a681998-2116-437f-831a-e26a564490d7" path="/var/lib/kubelet/pods/7a681998-2116-437f-831a-e26a564490d7/volumes"
Sep 10 21:32:49.098702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:49.098670    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"
Sep 10 21:32:49.098702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:49.098697    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4" event={"ID":"567256a5-37a1-40ab-8aa9-cfb06529c159","Type":"ContainerDied","Data":"51592a63ab0998ee9f7c2819539cde5ad84cf4fb661f8343b7fbae3ce6cc438c"}
Sep 10 21:32:49.099282 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:49.098745    2573 scope.go:117] "RemoveContainer" containerID="dfb0817e5398bcd6dd487fba2c766562d1dcb7bd315609d6bd6d99e7897b930c"
Sep 10 21:32:49.117627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:49.117598    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"]
Sep 10 21:32:49.120039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:49.120014    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/auth-test-model-1f3879-kserve-5d89cccfcf-mwmz4"]
Sep 10 21:32:50.679417 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:50.679386    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="567256a5-37a1-40ab-8aa9-cfb06529c159" path="/var/lib/kubelet/pods/567256a5-37a1-40ab-8aa9-cfb06529c159/volumes"
Sep 10 21:32:54.027282 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:54.027245    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:54.027282 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:54.027291    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:54.039924 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:54.039889    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:54.139449 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:54.139420    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:32:54.612462 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:32:54.612422    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:33:00.608795 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.608747    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"]
Sep 10 21:33:00.609271 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.609160    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a681998-2116-437f-831a-e26a564490d7" containerName="main"
Sep 10 21:33:00.609271 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.609173    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a681998-2116-437f-831a-e26a564490d7" containerName="main"
Sep 10 21:33:00.609271 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.609183    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="567256a5-37a1-40ab-8aa9-cfb06529c159" containerName="main"
Sep 10 21:33:00.609271 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.609191    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="567256a5-37a1-40ab-8aa9-cfb06529c159" containerName="main"
Sep 10 21:33:00.609407 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.609290    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a681998-2116-437f-831a-e26a564490d7" containerName="main"
Sep 10 21:33:00.609407 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.609306    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="567256a5-37a1-40ab-8aa9-cfb06529c159" containerName="main"
Sep 10 21:33:00.612767 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.612749    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.615813 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.615787    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"default-dockercfg-mf4gw\""
Sep 10 21:33:00.615813 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.615808    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"openshift-service-ca.crt\""
Sep 10 21:33:00.616016 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.615873    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"kube-root-ca.crt\""
Sep 10 21:33:00.616016 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.615793    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"sub-test-model-1f3879-kserve-self-signed-certs\""
Sep 10 21:33:00.624275 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.624251    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"]
Sep 10 21:33:00.726986 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.726950    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jznxv\" (UniqueName: \"kubernetes.io/projected/852a9889-8893-432b-bcd2-c7062ea688b6-kube-api-access-jznxv\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.727202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.727007    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-model-cache\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.727202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.727029    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-home\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.727202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.727076    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/852a9889-8893-432b-bcd2-c7062ea688b6-tls-certs\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.727202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.727100    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-tmp-dir\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.727202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.727123    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-dshm\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.827655 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.827616    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jznxv\" (UniqueName: \"kubernetes.io/projected/852a9889-8893-432b-bcd2-c7062ea688b6-kube-api-access-jznxv\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.827876 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.827684    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-model-cache\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.827876 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.827722    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-home\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.827876 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.827767    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/852a9889-8893-432b-bcd2-c7062ea688b6-tls-certs\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.827876 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.827842    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-tmp-dir\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.827876 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.827871    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-dshm\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.828213 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.828186    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-model-cache\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.828284 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.828213    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-home\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.828344 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.828320    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-tmp-dir\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.830223 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.830202    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-dshm\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.830344 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.830326    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/852a9889-8893-432b-bcd2-c7062ea688b6-tls-certs\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.835943 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.835924    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jznxv\" (UniqueName: \"kubernetes.io/projected/852a9889-8893-432b-bcd2-c7062ea688b6-kube-api-access-jznxv\") pod \"sub-test-model-1f3879-kserve-76596589c9-xj5hp\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:00.922031 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:00.921927    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:01.198568 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:01.198538    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"]
Sep 10 21:33:01.200656 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:33:01.200626    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod852a9889_8893_432b_bcd2_c7062ea688b6.slice/crio-43e5c1558f65a824851f50f197b7a20396e818f5e36c8f39b54508f155fa7224 WatchSource:0}: Error finding container 43e5c1558f65a824851f50f197b7a20396e818f5e36c8f39b54508f155fa7224: Status 404 returned error can't find the container with id 43e5c1558f65a824851f50f197b7a20396e818f5e36c8f39b54508f155fa7224
Sep 10 21:33:02.176119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:02.176069    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp" event={"ID":"852a9889-8893-432b-bcd2-c7062ea688b6","Type":"ContainerStarted","Data":"5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c"}
Sep 10 21:33:02.176119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:02.176124    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp" event={"ID":"852a9889-8893-432b-bcd2-c7062ea688b6","Type":"ContainerStarted","Data":"43e5c1558f65a824851f50f197b7a20396e818f5e36c8f39b54508f155fa7224"}
Sep 10 21:33:02.199301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:02.199250    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp" podStartSLOduration=2.19923495 podStartE2EDuration="2.19923495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:02.196763 +0000 UTC m=+1358.130429814" watchObservedRunningTime="2026-09-10 21:33:02.19923495 +0000 UTC m=+1358.132901837"
Sep 10 21:33:05.203853 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.203819    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"]
Sep 10 21:33:05.207509 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.207486    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.209849 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.209825    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-77a957-openshift-default-dockercfg-xg4sn\""
Sep 10 21:33:05.220411 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.220389    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"]
Sep 10 21:33:05.376566 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.376516    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdrct\" (UniqueName: \"kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-kube-api-access-kdrct\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.376749 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.376584    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-podinfo\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.376749 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.376638    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-certs\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.376749 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.376708    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ed944e7-d060-4ada-9637-048d564bb9c5-istiod-ca-cert\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.376886 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.376761    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-credential-socket\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.376886 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.376838    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-token\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.376886 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.376862    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-envoy\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.376989 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.376916    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-socket\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.376989 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.376956    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-data\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.478513 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.478410    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-socket\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.478513 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.478473    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-data\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.478513 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.478517    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdrct\" (UniqueName: \"kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-kube-api-access-kdrct\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.478754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.478547    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-podinfo\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.478830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.478742    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-certs\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.478897 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.478820    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ed944e7-d060-4ada-9637-048d564bb9c5-istiod-ca-cert\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.478897 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.478883    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-credential-socket\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.478897 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.478870    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-socket\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.479054 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.478925    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-token\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.479054 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.478968    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-envoy\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.479054 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.479011    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-data\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.479205 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.479083    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-certs\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.479460 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.479438    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-credential-socket\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.479556 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.479472    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ed944e7-d060-4ada-9637-048d564bb9c5-istiod-ca-cert\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.481063 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.481040    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-envoy\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.481162 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.481065    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-podinfo\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.486146 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.486122    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-token\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.486556 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.486533    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdrct\" (UniqueName: \"kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-kube-api-access-kdrct\") pod \"e2e-api-a-77a957-openshift-default-7d8869784-jkcjr\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") " pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.519411 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.519384    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:05.794377 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.794346    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"]
Sep 10 21:33:05.795013 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:33:05.794986    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed944e7_d060_4ada_9637_048d564bb9c5.slice/crio-2b921a60ef943802cb7610e9b179842ad8b8b64150b38bebdd0b918657c8f0ca WatchSource:0}: Error finding container 2b921a60ef943802cb7610e9b179842ad8b8b64150b38bebdd0b918657c8f0ca: Status 404 returned error can't find the container with id 2b921a60ef943802cb7610e9b179842ad8b8b64150b38bebdd0b918657c8f0ca
Sep 10 21:33:05.797440 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.797407    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:05.797537 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.797470    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:05.797537 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:05.797496    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:06.200458 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:06.200421    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" event={"ID":"4ed944e7-d060-4ada-9637-048d564bb9c5","Type":"ContainerStarted","Data":"ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6"}
Sep 10 21:33:06.200686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:06.200480    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" event={"ID":"4ed944e7-d060-4ada-9637-048d564bb9c5","Type":"ContainerStarted","Data":"2b921a60ef943802cb7610e9b179842ad8b8b64150b38bebdd0b918657c8f0ca"}
Sep 10 21:33:06.222668 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:06.222621    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" podStartSLOduration=1.222605438 podStartE2EDuration="1.222605438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:06.219720182 +0000 UTC m=+1362.153386976" watchObservedRunningTime="2026-09-10 21:33:06.222605438 +0000 UTC m=+1362.156272252"
Sep 10 21:33:06.519990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:06.519907    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:06.525314 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:06.525291    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:07.206567 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:07.206534    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:07.207481 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:07.207458    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:08.727112 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:08.727078    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:33:10.922576 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:10.922539    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:10.923043 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:10.922595    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:10.935242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:10.935212    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:11.241532 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:11.241444    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:33:26.329684 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:26.329647    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:33:27.341626 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:27.341592    2573 generic.go:356] "Generic (PLEG): container finished" podID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerID="ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6" exitCode=0
Sep 10 21:33:27.342084 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:27.341634    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" event={"ID":"4ed944e7-d060-4ada-9637-048d564bb9c5","Type":"ContainerDied","Data":"ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6"}
Sep 10 21:33:27.342129 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:27.342117    2573 scope.go:117] "RemoveContainer" containerID="ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6"
Sep 10 21:33:27.343236 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:27.343207    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:27.343348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:27.343289    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:27.343348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:27.343337    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:28.349379 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:28.349347    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" event={"ID":"4ed944e7-d060-4ada-9637-048d564bb9c5","Type":"ContainerStarted","Data":"595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2"}
Sep 10 21:33:28.520346 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:28.520308    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:28.520527 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:28.520506    2573 patch_prober.go:28] interesting pod/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 10 21:33:28.520580 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:28.520561    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 10 21:33:29.520051 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:29.520019    2573 patch_prober.go:28] interesting pod/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 10 21:33:29.520453 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:29.520077    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 10 21:33:30.520645 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:30.520601    2573 patch_prober.go:28] interesting pod/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 10 21:33:30.521143 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:30.520666    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 10 21:33:31.520499 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:31.520464    2573 patch_prober.go:28] interesting pod/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 10 21:33:31.520823 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:31.520521    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 10 21:33:32.520465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:32.520426    2573 patch_prober.go:28] interesting pod/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 10 21:33:32.520643 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:32.520514    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 10 21:33:33.581399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.581363    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:33.581891 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.581805    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:33.582631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.582613    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:33:33.761914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.761884    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"]
Sep 10 21:33:33.765814 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.765789    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.768566 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.768545    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-c71bd0-openshift-default-dockercfg-8kzn6\""
Sep 10 21:33:33.780048 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.780021    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"]
Sep 10 21:33:33.857084 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.856979    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-certs\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.857084 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.857016    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-token\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.857084 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.857046    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gtt2\" (UniqueName: \"kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-kube-api-access-9gtt2\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.857393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.857131    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-credential-socket\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.857393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.857162    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-envoy\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.857393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.857240    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-data\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.857393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.857268    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istiod-ca-cert\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.857393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.857316    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-podinfo\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.857393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.857355    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-socket\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.958590 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.958553    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-data\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.958770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.958599    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istiod-ca-cert\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.958770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.958652    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-podinfo\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.958770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.958684    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-socket\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.958770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.958745    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-certs\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.958770 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.958766    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-token\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.959050 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.958826    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gtt2\" (UniqueName: \"kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-kube-api-access-9gtt2\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.959050 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.958879    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-credential-socket\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.959050 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.958908    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-envoy\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.959213 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.959075    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-data\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.959213 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.959188    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-socket\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.959311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.959216    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-certs\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.959374 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.959355    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-credential-socket\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.959640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.959610    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istiod-ca-cert\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.961281 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.961260    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-envoy\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.961488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.961474    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-podinfo\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.966997 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.966974    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-token\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:33.966997 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:33.966991    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gtt2\" (UniqueName: \"kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-kube-api-access-9gtt2\") pod \"e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") " pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:34.075983 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:34.075945    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:34.352142 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:34.352112    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"]
Sep 10 21:33:34.353060 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:33:34.353037    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee4f0c7_4eaa_4753_96e4_911c26186cf7.slice/crio-b3402d1408ca329b6f665aa522bf4765feaad34e585ff021dc212344f03734df WatchSource:0}: Error finding container b3402d1408ca329b6f665aa522bf4765feaad34e585ff021dc212344f03734df: Status 404 returned error can't find the container with id b3402d1408ca329b6f665aa522bf4765feaad34e585ff021dc212344f03734df
Sep 10 21:33:34.355451 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:34.355416    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:34.355539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:34.355490    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:34.355539 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:34.355518    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:34.391650 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:34.391613    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" event={"ID":"6ee4f0c7-4eaa-4753-96e4-911c26186cf7","Type":"ContainerStarted","Data":"b3402d1408ca329b6f665aa522bf4765feaad34e585ff021dc212344f03734df"}
Sep 10 21:33:35.399570 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:35.399528    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" event={"ID":"6ee4f0c7-4eaa-4753-96e4-911c26186cf7","Type":"ContainerStarted","Data":"797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d"}
Sep 10 21:33:35.419735 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:35.419674    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" podStartSLOduration=2.419657624 podStartE2EDuration="2.419657624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:35.419619712 +0000 UTC m=+1391.353286520" watchObservedRunningTime="2026-09-10 21:33:35.419657624 +0000 UTC m=+1391.353324439"
Sep 10 21:33:36.077052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:36.077014    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:36.082639 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:36.082614    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:36.405326 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:36.405231    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:36.406266 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:36.406241    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:33:41.738734 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:41.738698    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7794-efbb39"]
Sep 10 21:33:41.742460 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:41.742442    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-efbb39"
Sep 10 21:33:41.744742 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:41.744727    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dxcl2\""
Sep 10 21:33:41.751141 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:41.751120    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7794-efbb39"]
Sep 10 21:33:41.834371 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:41.834334    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8kbw\" (UniqueName: \"kubernetes.io/projected/eb204bc4-99d1-407b-8f2a-a0be6e418e2e-kube-api-access-t8kbw\") pod \"test-curl-7794-efbb39\" (UID: \"eb204bc4-99d1-407b-8f2a-a0be6e418e2e\") " pod="opendatahub/test-curl-7794-efbb39"
Sep 10 21:33:41.935213 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:41.935177    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8kbw\" (UniqueName: \"kubernetes.io/projected/eb204bc4-99d1-407b-8f2a-a0be6e418e2e-kube-api-access-t8kbw\") pod \"test-curl-7794-efbb39\" (UID: \"eb204bc4-99d1-407b-8f2a-a0be6e418e2e\") " pod="opendatahub/test-curl-7794-efbb39"
Sep 10 21:33:41.943886 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:41.943855    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8kbw\" (UniqueName: \"kubernetes.io/projected/eb204bc4-99d1-407b-8f2a-a0be6e418e2e-kube-api-access-t8kbw\") pod \"test-curl-7794-efbb39\" (UID: \"eb204bc4-99d1-407b-8f2a-a0be6e418e2e\") " pod="opendatahub/test-curl-7794-efbb39"
Sep 10 21:33:42.053832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:42.053721    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-efbb39"
Sep 10 21:33:42.327424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:42.327393    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7794-efbb39"]
Sep 10 21:33:42.328877 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:33:42.328845    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb204bc4_99d1_407b_8f2a_a0be6e418e2e.slice/crio-8f58d77114ba03f70da2ce11c53e5835b02f6f039c35f37025130ffb8d633189 WatchSource:0}: Error finding container 8f58d77114ba03f70da2ce11c53e5835b02f6f039c35f37025130ffb8d633189: Status 404 returned error can't find the container with id 8f58d77114ba03f70da2ce11c53e5835b02f6f039c35f37025130ffb8d633189
Sep 10 21:33:42.441765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:42.441732    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7794-efbb39" event={"ID":"eb204bc4-99d1-407b-8f2a-a0be6e418e2e","Type":"ContainerStarted","Data":"2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa"}
Sep 10 21:33:42.441765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:42.441788    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7794-efbb39" event={"ID":"eb204bc4-99d1-407b-8f2a-a0be6e418e2e","Type":"ContainerStarted","Data":"8f58d77114ba03f70da2ce11c53e5835b02f6f039c35f37025130ffb8d633189"}
Sep 10 21:33:42.459055 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:42.458996    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7794-efbb39" podStartSLOduration=1.4589797 podStartE2EDuration="1.4589797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:42.456145774 +0000 UTC m=+1398.389812590" watchObservedRunningTime="2026-09-10 21:33:42.4589797 +0000 UTC m=+1398.392646553"
Sep 10 21:33:43.027184 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.027095    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7794-efbb39"]
Sep 10 21:33:43.031473 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.031443    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7794-efbb39"]
Sep 10 21:33:43.147819 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.147764    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7794-9d561f"]
Sep 10 21:33:43.148222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.148206    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb204bc4-99d1-407b-8f2a-a0be6e418e2e" containerName="test-curl-7794-efbb39"
Sep 10 21:33:43.148279 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.148225    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb204bc4-99d1-407b-8f2a-a0be6e418e2e" containerName="test-curl-7794-efbb39"
Sep 10 21:33:43.148318 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.148304    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb204bc4-99d1-407b-8f2a-a0be6e418e2e" containerName="test-curl-7794-efbb39"
Sep 10 21:33:43.151473 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.151458    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-9d561f"
Sep 10 21:33:43.158230 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.158200    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7794-9d561f"]
Sep 10 21:33:43.249100 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.249061    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfpdd\" (UniqueName: \"kubernetes.io/projected/9b68a82b-e8f0-4ba9-b51e-d96d0ad88844-kube-api-access-dfpdd\") pod \"test-curl-7794-9d561f\" (UID: \"9b68a82b-e8f0-4ba9-b51e-d96d0ad88844\") " pod="opendatahub/test-curl-7794-9d561f"
Sep 10 21:33:43.350301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.350266    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfpdd\" (UniqueName: \"kubernetes.io/projected/9b68a82b-e8f0-4ba9-b51e-d96d0ad88844-kube-api-access-dfpdd\") pod \"test-curl-7794-9d561f\" (UID: \"9b68a82b-e8f0-4ba9-b51e-d96d0ad88844\") " pod="opendatahub/test-curl-7794-9d561f"
Sep 10 21:33:43.358397 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.358376    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfpdd\" (UniqueName: \"kubernetes.io/projected/9b68a82b-e8f0-4ba9-b51e-d96d0ad88844-kube-api-access-dfpdd\") pod \"test-curl-7794-9d561f\" (UID: \"9b68a82b-e8f0-4ba9-b51e-d96d0ad88844\") " pod="opendatahub/test-curl-7794-9d561f"
Sep 10 21:33:43.447762 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:33:43.447730    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7794-efbb39" container="test-curl-7794-efbb39"
Sep 10 21:33:43.462558 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.462530    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-9d561f"
Sep 10 21:33:43.733984 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:43.733959    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7794-9d561f"]
Sep 10 21:33:43.735499 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:33:43.735474    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b68a82b_e8f0_4ba9_b51e_d96d0ad88844.slice/crio-86abf01e9513951bbd6e8d328056acd11b4ae6d74b0238fa94376c27b0f91e4f WatchSource:0}: Error finding container 86abf01e9513951bbd6e8d328056acd11b4ae6d74b0238fa94376c27b0f91e4f: Status 404 returned error can't find the container with id 86abf01e9513951bbd6e8d328056acd11b4ae6d74b0238fa94376c27b0f91e4f
Sep 10 21:33:44.457835 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:44.457801    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7794-9d561f" event={"ID":"9b68a82b-e8f0-4ba9-b51e-d96d0ad88844","Type":"ContainerStarted","Data":"ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59"}
Sep 10 21:33:44.458240 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:44.457843    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7794-9d561f" event={"ID":"9b68a82b-e8f0-4ba9-b51e-d96d0ad88844","Type":"ContainerStarted","Data":"86abf01e9513951bbd6e8d328056acd11b4ae6d74b0238fa94376c27b0f91e4f"}
Sep 10 21:33:44.458240 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:44.457903    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7794-efbb39" podUID="eb204bc4-99d1-407b-8f2a-a0be6e418e2e" containerName="test-curl-7794-efbb39" containerID="cri-o://2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa" gracePeriod=2
Sep 10 21:33:44.460053 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:33:44.460032    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7794-efbb39\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="eb204bc4-99d1-407b-8f2a-a0be6e418e2e" pod="opendatahub/test-curl-7794-efbb39"
Sep 10 21:33:44.475480 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:44.475439    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7794-9d561f" podStartSLOduration=1.475425395 podStartE2EDuration="1.475425395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:44.473948267 +0000 UTC m=+1400.407615081" watchObservedRunningTime="2026-09-10 21:33:44.475425395 +0000 UTC m=+1400.409092209"
Sep 10 21:33:44.475912 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:33:44.475890    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7794-efbb39\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="eb204bc4-99d1-407b-8f2a-a0be6e418e2e" pod="opendatahub/test-curl-7794-efbb39"
Sep 10 21:33:44.679108 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:33:44.679068    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7794-efbb39\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="eb204bc4-99d1-407b-8f2a-a0be6e418e2e" pod="opendatahub/test-curl-7794-efbb39"
Sep 10 21:33:45.027287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:45.027247    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7794-9d561f"]
Sep 10 21:33:45.033973 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:45.033937    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7794-9d561f"]
Sep 10 21:33:45.463925 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:33:45.463895    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7794-9d561f" container="test-curl-7794-9d561f"
Sep 10 21:33:46.469098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:46.469064    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7794-9d561f" podUID="9b68a82b-e8f0-4ba9-b51e-d96d0ad88844" containerName="test-curl-7794-9d561f" containerID="cri-o://ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59" gracePeriod=2
Sep 10 21:33:46.471130 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:33:46.471100    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7794-9d561f\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="9b68a82b-e8f0-4ba9-b51e-d96d0ad88844" pod="opendatahub/test-curl-7794-9d561f"
Sep 10 21:33:46.629750 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:46.629726    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-efbb39"
Sep 10 21:33:46.631571 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:33:46.631548    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7794-9d561f\" is forbidden: User \"system:node:ip-10-0-135-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-74.ec2.internal' and this object" podUID="9b68a82b-e8f0-4ba9-b51e-d96d0ad88844" pod="opendatahub/test-curl-7794-9d561f"
Sep 10 21:33:46.782378 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:46.782292    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8kbw\" (UniqueName: \"kubernetes.io/projected/eb204bc4-99d1-407b-8f2a-a0be6e418e2e-kube-api-access-t8kbw\") pod \"eb204bc4-99d1-407b-8f2a-a0be6e418e2e\" (UID: \"eb204bc4-99d1-407b-8f2a-a0be6e418e2e\") "
Sep 10 21:33:46.784424 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:46.784397    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb204bc4-99d1-407b-8f2a-a0be6e418e2e-kube-api-access-t8kbw" (OuterVolumeSpecName: "kube-api-access-t8kbw") pod "eb204bc4-99d1-407b-8f2a-a0be6e418e2e" (UID: "eb204bc4-99d1-407b-8f2a-a0be6e418e2e"). InnerVolumeSpecName "kube-api-access-t8kbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:33:46.883459 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:46.883413    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8kbw\" (UniqueName: \"kubernetes.io/projected/eb204bc4-99d1-407b-8f2a-a0be6e418e2e-kube-api-access-t8kbw\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:33:46.996563 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:46.994943    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"]
Sep 10 21:33:47.477888 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:47.477850    2573 generic.go:356] "Generic (PLEG): container finished" podID="eb204bc4-99d1-407b-8f2a-a0be6e418e2e" containerID="2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa" exitCode=137
Sep 10 21:33:47.478304 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:47.477903    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-efbb39"
Sep 10 21:33:47.478304 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:47.477916    2573 scope.go:117] "RemoveContainer" containerID="2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa"
Sep 10 21:33:47.488625 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:47.488607    2573 scope.go:117] "RemoveContainer" containerID="2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa"
Sep 10 21:33:47.488974 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:33:47.488951    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa\": container with ID starting with 2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa not found: ID does not exist" containerID="2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa"
Sep 10 21:33:47.489069 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:47.488979    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa"} err="failed to get container status \"2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa\": rpc error: code = NotFound desc = could not find container \"2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa\": container with ID starting with 2d0b4d15a99babdc4b21c08d45ea071b979a3e385b1d1c192940bab8c5cd29aa not found: ID does not exist"
Sep 10 21:33:47.996573 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:47.996532    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"]
Sep 10 21:33:48.480384 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:33:48.480353    2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b68a82b_e8f0_4ba9_b51e_d96d0ad88844.slice/crio-ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:33:48.644827 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:48.644798    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-9d561f"
Sep 10 21:33:48.679963 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:48.679926    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb204bc4-99d1-407b-8f2a-a0be6e418e2e" path="/var/lib/kubelet/pods/eb204bc4-99d1-407b-8f2a-a0be6e418e2e/volumes"
Sep 10 21:33:48.805711 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:48.805611    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfpdd\" (UniqueName: \"kubernetes.io/projected/9b68a82b-e8f0-4ba9-b51e-d96d0ad88844-kube-api-access-dfpdd\") pod \"9b68a82b-e8f0-4ba9-b51e-d96d0ad88844\" (UID: \"9b68a82b-e8f0-4ba9-b51e-d96d0ad88844\") "
Sep 10 21:33:48.807788 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:48.807753    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b68a82b-e8f0-4ba9-b51e-d96d0ad88844-kube-api-access-dfpdd" (OuterVolumeSpecName: "kube-api-access-dfpdd") pod "9b68a82b-e8f0-4ba9-b51e-d96d0ad88844" (UID: "9b68a82b-e8f0-4ba9-b51e-d96d0ad88844"). InnerVolumeSpecName "kube-api-access-dfpdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:33:48.907248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:48.907209    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfpdd\" (UniqueName: \"kubernetes.io/projected/9b68a82b-e8f0-4ba9-b51e-d96d0ad88844-kube-api-access-dfpdd\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:33:49.496171 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:49.496126    2573 generic.go:356] "Generic (PLEG): container finished" podID="9b68a82b-e8f0-4ba9-b51e-d96d0ad88844" containerID="ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59" exitCode=137
Sep 10 21:33:49.496643 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:49.496206    2573 scope.go:117] "RemoveContainer" containerID="ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59"
Sep 10 21:33:49.496643 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:49.496308    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7794-9d561f"
Sep 10 21:33:49.519731 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:49.519678    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:33:49.520707 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:49.520677    2573 scope.go:117] "RemoveContainer" containerID="ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59"
Sep 10 21:33:49.521172 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:33:49.521125    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59\": container with ID starting with ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59 not found: ID does not exist" containerID="ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59"
Sep 10 21:33:49.521294 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:49.521181    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59"} err="failed to get container status \"ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59\": rpc error: code = NotFound desc = could not find container \"ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59\": container with ID starting with ecadbde315217d5bf33bc128801a0a495660e9a9b143f466ad96659f90fe3b59 not found: ID does not exist"
Sep 10 21:33:50.678927 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:50.678893    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b68a82b-e8f0-4ba9-b51e-d96d0ad88844" path="/var/lib/kubelet/pods/9b68a82b-e8f0-4ba9-b51e-d96d0ad88844/volumes"
Sep 10 21:33:55.817476 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.817432    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"]
Sep 10 21:33:55.817906 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.817891    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b68a82b-e8f0-4ba9-b51e-d96d0ad88844" containerName="test-curl-7794-9d561f"
Sep 10 21:33:55.817973 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.817907    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b68a82b-e8f0-4ba9-b51e-d96d0ad88844" containerName="test-curl-7794-9d561f"
Sep 10 21:33:55.818007 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.817993    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b68a82b-e8f0-4ba9-b51e-d96d0ad88844" containerName="test-curl-7794-9d561f"
Sep 10 21:33:55.821000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.820983    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.823121 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.823097    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-36b938\"/\"rate-test-model-36b938-kserve-self-signed-certs\""
Sep 10 21:33:55.830766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.830740    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"]
Sep 10 21:33:55.895800 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.893265    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-dshm\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.895800 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.893415    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-home\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.895800 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.893453    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnt2c\" (UniqueName: \"kubernetes.io/projected/c8632dad-c12a-4af4-98ea-0c0a993ca59e-kube-api-access-qnt2c\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.895800 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.893500    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-model-cache\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.895800 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.893547    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tmp-dir\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.895800 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.893589    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tls-certs\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.994977 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.994930    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-home\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.995291 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.995266    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnt2c\" (UniqueName: \"kubernetes.io/projected/c8632dad-c12a-4af4-98ea-0c0a993ca59e-kube-api-access-qnt2c\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.995457 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.995436    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-model-cache\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.995601 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.995546    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-home\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.995601 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.995583    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tmp-dir\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.995756 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.995628    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tls-certs\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.995756 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.995664    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-dshm\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.995915 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.995881    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-model-cache\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.996196 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.996173    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tmp-dir\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.998215 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.998181    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-dshm\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:55.998355 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:55.998334    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tls-certs\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:56.003224 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:56.003200    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnt2c\" (UniqueName: \"kubernetes.io/projected/c8632dad-c12a-4af4-98ea-0c0a993ca59e-kube-api-access-qnt2c\") pod \"rate-test-model-36b938-kserve-6f6bb69595-5pr2n\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") " pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:56.131244 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:56.131209    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:33:56.426018 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:56.425992    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"]
Sep 10 21:33:56.427939 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:33:56.427910    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8632dad_c12a_4af4_98ea_0c0a993ca59e.slice/crio-cd4014b095fe628ad9f9d53a8a19e863c4d72e0531ec29ff997a706e9b56f3d3 WatchSource:0}: Error finding container cd4014b095fe628ad9f9d53a8a19e863c4d72e0531ec29ff997a706e9b56f3d3: Status 404 returned error can't find the container with id cd4014b095fe628ad9f9d53a8a19e863c4d72e0531ec29ff997a706e9b56f3d3
Sep 10 21:33:56.549109 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:56.549074    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n" event={"ID":"c8632dad-c12a-4af4-98ea-0c0a993ca59e","Type":"ContainerStarted","Data":"0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773"}
Sep 10 21:33:56.549282 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:56.549118    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n" event={"ID":"c8632dad-c12a-4af4-98ea-0c0a993ca59e","Type":"ContainerStarted","Data":"cd4014b095fe628ad9f9d53a8a19e863c4d72e0531ec29ff997a706e9b56f3d3"}
Sep 10 21:33:56.567466 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:56.567414    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n" podStartSLOduration=1.567386842 podStartE2EDuration="1.567386842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:56.566400326 +0000 UTC m=+1412.500067144" watchObservedRunningTime="2026-09-10 21:33:56.567386842 +0000 UTC m=+1412.501053658"
Sep 10 21:33:59.577064 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:59.577029    2573 generic.go:356] "Generic (PLEG): container finished" podID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerID="797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d" exitCode=0
Sep 10 21:33:59.577492 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:59.577102    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" event={"ID":"6ee4f0c7-4eaa-4753-96e4-911c26186cf7","Type":"ContainerDied","Data":"797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d"}
Sep 10 21:33:59.577492 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:59.577483    2573 scope.go:117] "RemoveContainer" containerID="797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d"
Sep 10 21:33:59.578645 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:59.578620    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:59.578702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:59.578690    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:33:59.578741 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:33:59.578725    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:34:00.586414 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:00.586383    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" event={"ID":"6ee4f0c7-4eaa-4753-96e4-911c26186cf7","Type":"ContainerStarted","Data":"10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef"}
Sep 10 21:34:01.076525 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:01.076480    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:34:01.076878 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:01.076840    2573 patch_prober.go:28] interesting pod/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 10 21:34:01.076974 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:01.076943    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 10 21:34:02.020111 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.020069    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t" podUID="2ef9e793-69e7-459d-a45c-320f73717f23" containerName="main" containerID="cri-o://47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b" gracePeriod=45
Sep 10 21:34:02.077026 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.076983    2573 patch_prober.go:28] interesting pod/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 10 21:34:02.077225 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.077059    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 10 21:34:02.309548 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.309524    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:34:02.358072 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358041    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ef9e793-69e7-459d-a45c-320f73717f23-tls-certs\") pod \"2ef9e793-69e7-459d-a45c-320f73717f23\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") "
Sep 10 21:34:02.358072 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358082    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-dshm\") pod \"2ef9e793-69e7-459d-a45c-320f73717f23\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") "
Sep 10 21:34:02.358331 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358129    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-tmp-dir\") pod \"2ef9e793-69e7-459d-a45c-320f73717f23\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") "
Sep 10 21:34:02.358331 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358146    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-model-cache\") pod \"2ef9e793-69e7-459d-a45c-320f73717f23\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") "
Sep 10 21:34:02.358331 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358225    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-home\") pod \"2ef9e793-69e7-459d-a45c-320f73717f23\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") "
Sep 10 21:34:02.358331 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358296    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89bfl\" (UniqueName: \"kubernetes.io/projected/2ef9e793-69e7-459d-a45c-320f73717f23-kube-api-access-89bfl\") pod \"2ef9e793-69e7-459d-a45c-320f73717f23\" (UID: \"2ef9e793-69e7-459d-a45c-320f73717f23\") "
Sep 10 21:34:02.358553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358393    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2ef9e793-69e7-459d-a45c-320f73717f23" (UID: "2ef9e793-69e7-459d-a45c-320f73717f23"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:02.358553 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358493    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-model-cache" (OuterVolumeSpecName: "model-cache") pod "2ef9e793-69e7-459d-a45c-320f73717f23" (UID: "2ef9e793-69e7-459d-a45c-320f73717f23"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:02.358644 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358540    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-home" (OuterVolumeSpecName: "home") pod "2ef9e793-69e7-459d-a45c-320f73717f23" (UID: "2ef9e793-69e7-459d-a45c-320f73717f23"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:02.358730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358703    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:02.358805 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358731    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:02.358805 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.358745    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:02.360444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.360410    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ef9e793-69e7-459d-a45c-320f73717f23-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2ef9e793-69e7-459d-a45c-320f73717f23" (UID: "2ef9e793-69e7-459d-a45c-320f73717f23"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:34:02.360444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.360412    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ef9e793-69e7-459d-a45c-320f73717f23-kube-api-access-89bfl" (OuterVolumeSpecName: "kube-api-access-89bfl") pod "2ef9e793-69e7-459d-a45c-320f73717f23" (UID: "2ef9e793-69e7-459d-a45c-320f73717f23"). InnerVolumeSpecName "kube-api-access-89bfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:02.360640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.360488    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-dshm" (OuterVolumeSpecName: "dshm") pod "2ef9e793-69e7-459d-a45c-320f73717f23" (UID: "2ef9e793-69e7-459d-a45c-320f73717f23"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:02.459617 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.459573    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89bfl\" (UniqueName: \"kubernetes.io/projected/2ef9e793-69e7-459d-a45c-320f73717f23-kube-api-access-89bfl\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:02.459617 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.459610    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ef9e793-69e7-459d-a45c-320f73717f23-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:02.459617 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.459622    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ef9e793-69e7-459d-a45c-320f73717f23-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:02.604988 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.604952    2573 generic.go:356] "Generic (PLEG): container finished" podID="2ef9e793-69e7-459d-a45c-320f73717f23" containerID="47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b" exitCode=0
Sep 10 21:34:02.605197 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.605007    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"
Sep 10 21:34:02.605197 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.605043    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t" event={"ID":"2ef9e793-69e7-459d-a45c-320f73717f23","Type":"ContainerDied","Data":"47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b"}
Sep 10 21:34:02.605197 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.605091    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t" event={"ID":"2ef9e793-69e7-459d-a45c-320f73717f23","Type":"ContainerDied","Data":"2f3e33882ba2c10731cf31ab684231306f95703d7595f6f88a128e48c21bcb0b"}
Sep 10 21:34:02.605197 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.605113    2573 scope.go:117] "RemoveContainer" containerID="47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b"
Sep 10 21:34:02.615094 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.615076    2573 scope.go:117] "RemoveContainer" containerID="47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b"
Sep 10 21:34:02.615351 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:34:02.615335    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b\": container with ID starting with 47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b not found: ID does not exist" containerID="47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b"
Sep 10 21:34:02.615396 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.615358    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b"} err="failed to get container status \"47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b\": rpc error: code = NotFound desc = could not find container \"47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b\": container with ID starting with 47b3a1d5680cd71a630f297087f669973058b8fd192ee6bb7e376d8758ef1f5b not found: ID does not exist"
Sep 10 21:34:02.631337 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.631310    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"]
Sep 10 21:34:02.635408 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.635386    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/sub-test-model-36b938-kserve-897946849-z7v2t"]
Sep 10 21:34:02.695259 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:02.695217    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ef9e793-69e7-459d-a45c-320f73717f23" path="/var/lib/kubelet/pods/2ef9e793-69e7-459d-a45c-320f73717f23/volumes"
Sep 10 21:34:03.015582 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.015495    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp" podUID="852a9889-8893-432b-bcd2-c7062ea688b6" containerName="main" containerID="cri-o://5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c" gracePeriod=45
Sep 10 21:34:03.077157 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.076683    2573 patch_prober.go:28] interesting pod/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 10 21:34:03.077157 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.076745    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 10 21:34:03.303456 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.303427    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:34:03.369304 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.369265    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-dshm\") pod \"852a9889-8893-432b-bcd2-c7062ea688b6\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") "
Sep 10 21:34:03.369304 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.369301    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-model-cache\") pod \"852a9889-8893-432b-bcd2-c7062ea688b6\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") "
Sep 10 21:34:03.369517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.369375    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/852a9889-8893-432b-bcd2-c7062ea688b6-tls-certs\") pod \"852a9889-8893-432b-bcd2-c7062ea688b6\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") "
Sep 10 21:34:03.369517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.369418    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jznxv\" (UniqueName: \"kubernetes.io/projected/852a9889-8893-432b-bcd2-c7062ea688b6-kube-api-access-jznxv\") pod \"852a9889-8893-432b-bcd2-c7062ea688b6\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") "
Sep 10 21:34:03.369517 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.369467    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-tmp-dir\") pod \"852a9889-8893-432b-bcd2-c7062ea688b6\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") "
Sep 10 21:34:03.369673 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.369588    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-model-cache" (OuterVolumeSpecName: "model-cache") pod "852a9889-8893-432b-bcd2-c7062ea688b6" (UID: "852a9889-8893-432b-bcd2-c7062ea688b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:03.369673 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.369604    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-home\") pod \"852a9889-8893-432b-bcd2-c7062ea688b6\" (UID: \"852a9889-8893-432b-bcd2-c7062ea688b6\") "
Sep 10 21:34:03.369797 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.369742    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "852a9889-8893-432b-bcd2-c7062ea688b6" (UID: "852a9889-8893-432b-bcd2-c7062ea688b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:03.369860 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.369809    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-home" (OuterVolumeSpecName: "home") pod "852a9889-8893-432b-bcd2-c7062ea688b6" (UID: "852a9889-8893-432b-bcd2-c7062ea688b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:03.370083 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.370064    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:03.370152 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.370088    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:03.370152 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.370101    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:03.388288 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.388236    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/852a9889-8893-432b-bcd2-c7062ea688b6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "852a9889-8893-432b-bcd2-c7062ea688b6" (UID: "852a9889-8893-432b-bcd2-c7062ea688b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:34:03.391801 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.390901    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/852a9889-8893-432b-bcd2-c7062ea688b6-kube-api-access-jznxv" (OuterVolumeSpecName: "kube-api-access-jznxv") pod "852a9889-8893-432b-bcd2-c7062ea688b6" (UID: "852a9889-8893-432b-bcd2-c7062ea688b6"). InnerVolumeSpecName "kube-api-access-jznxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:03.392925 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.392887    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-dshm" (OuterVolumeSpecName: "dshm") pod "852a9889-8893-432b-bcd2-c7062ea688b6" (UID: "852a9889-8893-432b-bcd2-c7062ea688b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:03.481523 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.481470    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/852a9889-8893-432b-bcd2-c7062ea688b6-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:03.481523 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.481523    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jznxv\" (UniqueName: \"kubernetes.io/projected/852a9889-8893-432b-bcd2-c7062ea688b6-kube-api-access-jznxv\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:03.481832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.481539    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/852a9889-8893-432b-bcd2-c7062ea688b6-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:03.611937 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.611903    2573 generic.go:356] "Generic (PLEG): container finished" podID="852a9889-8893-432b-bcd2-c7062ea688b6" containerID="5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c" exitCode=0
Sep 10 21:34:03.612141 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.611946    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"
Sep 10 21:34:03.612141 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.611986    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp" event={"ID":"852a9889-8893-432b-bcd2-c7062ea688b6","Type":"ContainerDied","Data":"5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c"}
Sep 10 21:34:03.612141 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.612037    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp" event={"ID":"852a9889-8893-432b-bcd2-c7062ea688b6","Type":"ContainerDied","Data":"43e5c1558f65a824851f50f197b7a20396e818f5e36c8f39b54508f155fa7224"}
Sep 10 21:34:03.612141 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.612060    2573 scope.go:117] "RemoveContainer" containerID="5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c"
Sep 10 21:34:03.621062 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.621046    2573 scope.go:117] "RemoveContainer" containerID="5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c"
Sep 10 21:34:03.621320 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:34:03.621303    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c\": container with ID starting with 5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c not found: ID does not exist" containerID="5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c"
Sep 10 21:34:03.621397 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.621327    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c"} err="failed to get container status \"5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c\": rpc error: code = NotFound desc = could not find container \"5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c\": container with ID starting with 5c2201a14ee4e5b4be5231d2a61f2b3f9a1b792346f79cb7841ae31b78cb583c not found: ID does not exist"
Sep 10 21:34:03.634990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.634965    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"]
Sep 10 21:34:03.640708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:03.640686    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/sub-test-model-1f3879-kserve-76596589c9-xj5hp"]
Sep 10 21:34:04.077215 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.076696    2573 patch_prober.go:28] interesting pod/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 10 21:34:04.077215 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.076799    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 10 21:34:04.683226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.683188    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="852a9889-8893-432b-bcd2-c7062ea688b6" path="/var/lib/kubelet/pods/852a9889-8893-432b-bcd2-c7062ea688b6/volumes"
Sep 10 21:34:04.941519 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.941434    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"]
Sep 10 21:34:04.941892 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.941878    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="852a9889-8893-432b-bcd2-c7062ea688b6" containerName="main"
Sep 10 21:34:04.941943 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.941894    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="852a9889-8893-432b-bcd2-c7062ea688b6" containerName="main"
Sep 10 21:34:04.941943 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.941906    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef9e793-69e7-459d-a45c-320f73717f23" containerName="main"
Sep 10 21:34:04.941943 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.941912    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef9e793-69e7-459d-a45c-320f73717f23" containerName="main"
Sep 10 21:34:04.942050 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.941983    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef9e793-69e7-459d-a45c-320f73717f23" containerName="main"
Sep 10 21:34:04.942050 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.941992    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="852a9889-8893-432b-bcd2-c7062ea688b6" containerName="main"
Sep 10 21:34:04.945135 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.945119    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"
Sep 10 21:34:04.947243 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.947226    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:34:04.951859 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.951837    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"]
Sep 10 21:34:04.999418 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:04.999378    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9bb364-0203-4fc5-969a-a4953a9542ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-77a957-pq5kw\" (UID: \"9e9bb364-0203-4fc5-969a-a4953a9542ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"
Sep 10 21:34:05.092802 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:05.088172    2573 patch_prober.go:28] interesting pod/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 10 21:34:05.092802 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:05.088250    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 10 21:34:05.100173 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:05.100144    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9bb364-0203-4fc5-969a-a4953a9542ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-77a957-pq5kw\" (UID: \"9e9bb364-0203-4fc5-969a-a4953a9542ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"
Sep 10 21:34:05.101052 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:05.101024    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9bb364-0203-4fc5-969a-a4953a9542ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-77a957-pq5kw\" (UID: \"9e9bb364-0203-4fc5-969a-a4953a9542ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"
Sep 10 21:34:05.257544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:05.257453    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"
Sep 10 21:34:05.558604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:05.558573    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"]
Sep 10 21:34:05.559807 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:34:05.559756    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e9bb364_0203_4fc5_969a_a4953a9542ff.slice/crio-d05f49dbe427f57bcd2f138f944b16b2ca1c2a367d5eed1d0bafb41d8ad4e964 WatchSource:0}: Error finding container d05f49dbe427f57bcd2f138f944b16b2ca1c2a367d5eed1d0bafb41d8ad4e964: Status 404 returned error can't find the container with id d05f49dbe427f57bcd2f138f944b16b2ca1c2a367d5eed1d0bafb41d8ad4e964
Sep 10 21:34:05.627011 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:05.626979    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw" event={"ID":"9e9bb364-0203-4fc5-969a-a4953a9542ff","Type":"ContainerStarted","Data":"d05f49dbe427f57bcd2f138f944b16b2ca1c2a367d5eed1d0bafb41d8ad4e964"}
Sep 10 21:34:06.077216 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:06.077170    2573 patch_prober.go:28] interesting pod/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 10 21:34:06.077430 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:06.077239    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 10 21:34:06.132381 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:06.132335    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:34:06.132869 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:06.132399    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:34:06.145053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:06.145025    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:34:06.406097 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:06.406060    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:34:06.634961 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:06.634931    2573 generic.go:356] "Generic (PLEG): container finished" podID="9e9bb364-0203-4fc5-969a-a4953a9542ff" containerID="5ae6d9cc18f6dccf93725b12acd8829b1cc965a0533ee1a2768a1f04e8723a98" exitCode=0
Sep 10 21:34:06.635139 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:06.635017    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw" event={"ID":"9e9bb364-0203-4fc5-969a-a4953a9542ff","Type":"ContainerDied","Data":"5ae6d9cc18f6dccf93725b12acd8829b1cc965a0533ee1a2768a1f04e8723a98"}
Sep 10 21:34:06.646593 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:06.646569    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:34:07.081282 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:07.081245    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:34:07.082253 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:07.082221    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:34:07.776035 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:07.776009    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"
Sep 10 21:34:07.826889 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:07.826860    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9bb364-0203-4fc5-969a-a4953a9542ff-maas-api-service-ca\") pod \"9e9bb364-0203-4fc5-969a-a4953a9542ff\" (UID: \"9e9bb364-0203-4fc5-969a-a4953a9542ff\") "
Sep 10 21:34:07.827253 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:07.827234    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e9bb364-0203-4fc5-969a-a4953a9542ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e9bb364-0203-4fc5-969a-a4953a9542ff" (UID: "9e9bb364-0203-4fc5-969a-a4953a9542ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:07.928252 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:07.928157    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9bb364-0203-4fc5-969a-a4953a9542ff-maas-api-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:08.648395 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:08.648366    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"
Sep 10 21:34:08.648578 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:08.648366    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw" event={"ID":"9e9bb364-0203-4fc5-969a-a4953a9542ff","Type":"ContainerDied","Data":"d05f49dbe427f57bcd2f138f944b16b2ca1c2a367d5eed1d0bafb41d8ad4e964"}
Sep 10 21:34:08.648578 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:08.648495    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d05f49dbe427f57bcd2f138f944b16b2ca1c2a367d5eed1d0bafb41d8ad4e964"
Sep 10 21:34:10.519925 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:10.519890    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:34:17.722548 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.722466    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"]
Sep 10 21:34:17.723071 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.723050    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e9bb364-0203-4fc5-969a-a4953a9542ff" containerName="revoke-keys"
Sep 10 21:34:17.723153 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.723074    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9bb364-0203-4fc5-969a-a4953a9542ff" containerName="revoke-keys"
Sep 10 21:34:17.723228 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.723215    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e9bb364-0203-4fc5-969a-a4953a9542ff" containerName="revoke-keys"
Sep 10 21:34:17.726508 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.726487    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.729754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.729733    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"default-dockercfg-mf4gw\""
Sep 10 21:34:17.729940 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.729800    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"kube-root-ca.crt\""
Sep 10 21:34:17.730048 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.729831    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"openshift-service-ca.crt\""
Sep 10 21:34:17.730107 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.729869    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1f3879\"/\"rate-test-model-1f3879-kserve-self-signed-certs\""
Sep 10 21:34:17.734195 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.734168    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"]
Sep 10 21:34:17.828051 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.828010    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-model-cache\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.828051 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.828055    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-dshm\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.828293 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.828144    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-tmp-dir\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.828293 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.828202    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j2wc\" (UniqueName: \"kubernetes.io/projected/7fff0526-6efe-4634-8b19-04b2e036a44c-kube-api-access-5j2wc\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.828293 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.828238    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fff0526-6efe-4634-8b19-04b2e036a44c-tls-certs\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.828293 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.828286    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-home\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.929633 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.929596    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j2wc\" (UniqueName: \"kubernetes.io/projected/7fff0526-6efe-4634-8b19-04b2e036a44c-kube-api-access-5j2wc\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.929832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.929640    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fff0526-6efe-4634-8b19-04b2e036a44c-tls-certs\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.929832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.929667    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-home\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.929832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.929727    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-model-cache\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.929832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.929751    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-dshm\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.930081 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.929836    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-tmp-dir\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.930217 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.930192    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-home\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.930354 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.930242    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-tmp-dir\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.930408 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.930366    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-model-cache\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.932081 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.932050    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-dshm\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.932488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.932464    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fff0526-6efe-4634-8b19-04b2e036a44c-tls-certs\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:17.938098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:17.938075    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j2wc\" (UniqueName: \"kubernetes.io/projected/7fff0526-6efe-4634-8b19-04b2e036a44c-kube-api-access-5j2wc\") pod \"rate-test-model-1f3879-kserve-5c8649459f-qqvks\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") " pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:18.039045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:18.038960    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:18.311214 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:18.311177    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"]
Sep 10 21:34:18.311745 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:34:18.311721    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fff0526_6efe_4634_8b19_04b2e036a44c.slice/crio-97640b5a30f05abbf6e5e19179f9de80c95012da3412355b87f9b6377e4340c6 WatchSource:0}: Error finding container 97640b5a30f05abbf6e5e19179f9de80c95012da3412355b87f9b6377e4340c6: Status 404 returned error can't find the container with id 97640b5a30f05abbf6e5e19179f9de80c95012da3412355b87f9b6377e4340c6
Sep 10 21:34:18.706511 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:18.706467    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks" event={"ID":"7fff0526-6efe-4634-8b19-04b2e036a44c","Type":"ContainerStarted","Data":"fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263"}
Sep 10 21:34:18.706720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:18.706518    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks" event={"ID":"7fff0526-6efe-4634-8b19-04b2e036a44c","Type":"ContainerStarted","Data":"97640b5a30f05abbf6e5e19179f9de80c95012da3412355b87f9b6377e4340c6"}
Sep 10 21:34:18.725173 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:18.725123    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks" podStartSLOduration=1.7251064820000002 podStartE2EDuration="1.725106482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:34:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:34:18.725082648 +0000 UTC m=+1434.658749464" watchObservedRunningTime="2026-09-10 21:34:18.725106482 +0000 UTC m=+1434.658773297"
Sep 10 21:34:20.606477 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:20.606439    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"]
Sep 10 21:34:20.611989 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:20.611962    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a957-pq5kw"]
Sep 10 21:34:20.680030 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:20.679992    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e9bb364-0203-4fc5-969a-a4953a9542ff" path="/var/lib/kubelet/pods/9e9bb364-0203-4fc5-969a-a4953a9542ff/volumes"
Sep 10 21:34:24.425370 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:24.425332    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:34:27.017410 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.017373    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"]
Sep 10 21:34:27.017918 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.017611    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy" containerID="cri-o://595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2" gracePeriod=30
Sep 10 21:34:27.357642 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.357607    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"]
Sep 10 21:34:27.360967 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.360951    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"
Sep 10 21:34:27.363308 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.363283    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:34:27.370589 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.370566    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"]
Sep 10 21:34:27.522547 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.522512    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb\" (UID: \"1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"
Sep 10 21:34:27.623962 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.623871    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb\" (UID: \"1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"
Sep 10 21:34:27.624506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.624484    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb\" (UID: \"1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"
Sep 10 21:34:27.672814 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.672756    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"
Sep 10 21:34:27.946932 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:27.946903    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"]
Sep 10 21:34:27.948369 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:34:27.948346    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bbdc468_9bda_47a4_a2a3_15ce18b1c0c5.slice/crio-7c9067f13b0d0ae8f3c1f11b54b961085f9140e7b9f84b2802e5287a52146346 WatchSource:0}: Error finding container 7c9067f13b0d0ae8f3c1f11b54b961085f9140e7b9f84b2802e5287a52146346: Status 404 returned error can't find the container with id 7c9067f13b0d0ae8f3c1f11b54b961085f9140e7b9f84b2802e5287a52146346
Sep 10 21:34:28.039220 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:28.039187    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:28.039661 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:28.039249    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:28.052133 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:28.052106    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:28.768991 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:28.768959    2573 generic.go:356] "Generic (PLEG): container finished" podID="1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5" containerID="f3103f6eb6ce84e4c073f71b118bc6b134b0c147fb0a6e7433010ea8aa1e7547" exitCode=0
Sep 10 21:34:28.769195 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:28.769024    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb" event={"ID":"1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5","Type":"ContainerDied","Data":"f3103f6eb6ce84e4c073f71b118bc6b134b0c147fb0a6e7433010ea8aa1e7547"}
Sep 10 21:34:28.769195 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:28.769086    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb" event={"ID":"1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5","Type":"ContainerStarted","Data":"7c9067f13b0d0ae8f3c1f11b54b961085f9140e7b9f84b2802e5287a52146346"}
Sep 10 21:34:28.780710 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:28.780683    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:34:29.898656 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:29.898630    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"
Sep 10 21:34:30.047967 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:30.047876    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5-maas-api-service-ca\") pod \"1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5\" (UID: \"1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5\") "
Sep 10 21:34:30.048246 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:30.048222    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5" (UID: "1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:30.149542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:30.149487    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5-maas-api-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:30.782202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:30.782167    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb" event={"ID":"1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5","Type":"ContainerDied","Data":"7c9067f13b0d0ae8f3c1f11b54b961085f9140e7b9f84b2802e5287a52146346"}
Sep 10 21:34:30.782202 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:30.782202    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"
Sep 10 21:34:30.782416 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:30.782218    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c9067f13b0d0ae8f3c1f11b54b961085f9140e7b9f84b2802e5287a52146346"
Sep 10 21:34:32.274902 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.274853    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:34:32.471641 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.471601    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-socket\") pod \"4ed944e7-d060-4ada-9637-048d564bb9c5\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") "
Sep 10 21:34:32.471859 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.471662    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdrct\" (UniqueName: \"kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-kube-api-access-kdrct\") pod \"4ed944e7-d060-4ada-9637-048d564bb9c5\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") "
Sep 10 21:34:32.471859 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.471747    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-certs\") pod \"4ed944e7-d060-4ada-9637-048d564bb9c5\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") "
Sep 10 21:34:32.471859 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.471797    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-token\") pod \"4ed944e7-d060-4ada-9637-048d564bb9c5\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") "
Sep 10 21:34:32.472004 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.471949    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ed944e7-d060-4ada-9637-048d564bb9c5" (UID: "4ed944e7-d060-4ada-9637-048d564bb9c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:32.472063 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472006    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-podinfo\") pod \"4ed944e7-d060-4ada-9637-048d564bb9c5\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") "
Sep 10 21:34:32.472119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472060    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-data\") pod \"4ed944e7-d060-4ada-9637-048d564bb9c5\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") "
Sep 10 21:34:32.472119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472085    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ed944e7-d060-4ada-9637-048d564bb9c5" (UID: "4ed944e7-d060-4ada-9637-048d564bb9c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:32.472119 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472095    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-credential-socket\") pod \"4ed944e7-d060-4ada-9637-048d564bb9c5\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") "
Sep 10 21:34:32.472287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472175    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-envoy\") pod \"4ed944e7-d060-4ada-9637-048d564bb9c5\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") "
Sep 10 21:34:32.472287 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472211    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ed944e7-d060-4ada-9637-048d564bb9c5-istiod-ca-cert\") pod \"4ed944e7-d060-4ada-9637-048d564bb9c5\" (UID: \"4ed944e7-d060-4ada-9637-048d564bb9c5\") "
Sep 10 21:34:32.472383 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472293    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ed944e7-d060-4ada-9637-048d564bb9c5" (UID: "4ed944e7-d060-4ada-9637-048d564bb9c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:32.472383 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472310    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ed944e7-d060-4ada-9637-048d564bb9c5" (UID: "4ed944e7-d060-4ada-9637-048d564bb9c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:32.472730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472646    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:32.472730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472673    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:32.472730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472687    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:32.472730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472687    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ed944e7-d060-4ada-9637-048d564bb9c5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ed944e7-d060-4ada-9637-048d564bb9c5" (UID: "4ed944e7-d060-4ada-9637-048d564bb9c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:32.472730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.472700    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:32.474073 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.474049    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-kube-api-access-kdrct" (OuterVolumeSpecName: "kube-api-access-kdrct") pod "4ed944e7-d060-4ada-9637-048d564bb9c5" (UID: "4ed944e7-d060-4ada-9637-048d564bb9c5"). InnerVolumeSpecName "kube-api-access-kdrct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:32.474254 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.474237    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ed944e7-d060-4ada-9637-048d564bb9c5" (UID: "4ed944e7-d060-4ada-9637-048d564bb9c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:32.474495 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.474474    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ed944e7-d060-4ada-9637-048d564bb9c5" (UID: "4ed944e7-d060-4ada-9637-048d564bb9c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:34:32.474581 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.474489    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ed944e7-d060-4ada-9637-048d564bb9c5" (UID: "4ed944e7-d060-4ada-9637-048d564bb9c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:32.573672 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.573637    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:32.573672 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.573668    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:32.573672 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.573677    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ed944e7-d060-4ada-9637-048d564bb9c5-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:32.573950 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.573684    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ed944e7-d060-4ada-9637-048d564bb9c5-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:32.573950 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.573693    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdrct\" (UniqueName: \"kubernetes.io/projected/4ed944e7-d060-4ada-9637-048d564bb9c5-kube-api-access-kdrct\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:32.795405 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.795317    2573 generic.go:356] "Generic (PLEG): container finished" podID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerID="595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2" exitCode=0
Sep 10 21:34:32.795405 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.795368    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"
Sep 10 21:34:32.795602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.795404    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" event={"ID":"4ed944e7-d060-4ada-9637-048d564bb9c5","Type":"ContainerDied","Data":"595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2"}
Sep 10 21:34:32.795602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.795457    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr" event={"ID":"4ed944e7-d060-4ada-9637-048d564bb9c5","Type":"ContainerDied","Data":"2b921a60ef943802cb7610e9b179842ad8b8b64150b38bebdd0b918657c8f0ca"}
Sep 10 21:34:32.795602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.795480    2573 scope.go:117] "RemoveContainer" containerID="595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2"
Sep 10 21:34:32.804842 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.804827    2573 scope.go:117] "RemoveContainer" containerID="ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6"
Sep 10 21:34:32.813863 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.813843    2573 scope.go:117] "RemoveContainer" containerID="595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2"
Sep 10 21:34:32.814143 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:34:32.814124    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2\": container with ID starting with 595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2 not found: ID does not exist" containerID="595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2"
Sep 10 21:34:32.814215 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.814153    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2"} err="failed to get container status \"595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2\": rpc error: code = NotFound desc = could not find container \"595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2\": container with ID starting with 595c7d38e21fda5a52f09dda3f9416e3fe0933b87bcafb18d4ede6263daf5ac2 not found: ID does not exist"
Sep 10 21:34:32.814215 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.814173    2573 scope.go:117] "RemoveContainer" containerID="ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6"
Sep 10 21:34:32.814459 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:34:32.814439    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6\": container with ID starting with ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6 not found: ID does not exist" containerID="ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6"
Sep 10 21:34:32.814519 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.814465    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6"} err="failed to get container status \"ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6\": rpc error: code = NotFound desc = could not find container \"ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6\": container with ID starting with ffc6188bb67903e31977f39737d795314f0c34ed530b12b84f428b890ffd33c6 not found: ID does not exist"
Sep 10 21:34:32.815982 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.815957    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"]
Sep 10 21:34:32.819658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:32.819637    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-77a957-openshift-default-7d8869784-jkcjr"]
Sep 10 21:34:34.680064 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:34.680032    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" path="/var/lib/kubelet/pods/4ed944e7-d060-4ada-9637-048d564bb9c5/volumes"
Sep 10 21:34:43.154228 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:43.154192    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"]
Sep 10 21:34:43.157369 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:43.157342    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c71bd0-bg7cb"]
Sep 10 21:34:44.680327 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:44.680290    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5" path="/var/lib/kubelet/pods/1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5/volumes"
Sep 10 21:34:46.718083 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:46.718046    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:34:49.669497 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:49.669461    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"]
Sep 10 21:34:49.669910 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:49.669736    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy" containerID="cri-o://10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef" gracePeriod=30
Sep 10 21:34:52.083555 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:52.083514    2573 patch_prober.go:28] interesting pod/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:34:52.083977 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:52.083594    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:34:54.924467 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.924433    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:34:54.939322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.939230    2573 generic.go:356] "Generic (PLEG): container finished" podID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerID="10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef" exitCode=0
Sep 10 21:34:54.939322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.939311    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"
Sep 10 21:34:54.939536 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.939313    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" event={"ID":"6ee4f0c7-4eaa-4753-96e4-911c26186cf7","Type":"ContainerDied","Data":"10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef"}
Sep 10 21:34:54.939536 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.939415    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd" event={"ID":"6ee4f0c7-4eaa-4753-96e4-911c26186cf7","Type":"ContainerDied","Data":"b3402d1408ca329b6f665aa522bf4765feaad34e585ff021dc212344f03734df"}
Sep 10 21:34:54.939536 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.939450    2573 scope.go:117] "RemoveContainer" containerID="10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef"
Sep 10 21:34:54.949683 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.949661    2573 scope.go:117] "RemoveContainer" containerID="797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d"
Sep 10 21:34:54.958358 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.958337    2573 scope.go:117] "RemoveContainer" containerID="10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef"
Sep 10 21:34:54.958676 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:34:54.958652    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef\": container with ID starting with 10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef not found: ID does not exist" containerID="10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef"
Sep 10 21:34:54.958735 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.958685    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef"} err="failed to get container status \"10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef\": rpc error: code = NotFound desc = could not find container \"10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef\": container with ID starting with 10e5c17d51471f56da015b16cf86a24afb7b8c1e2be71385367c702d51f4e7ef not found: ID does not exist"
Sep 10 21:34:54.958735 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.958704    2573 scope.go:117] "RemoveContainer" containerID="797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d"
Sep 10 21:34:54.958998 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:34:54.958980    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d\": container with ID starting with 797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d not found: ID does not exist" containerID="797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d"
Sep 10 21:34:54.959055 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.959004    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d"} err="failed to get container status \"797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d\": rpc error: code = NotFound desc = could not find container \"797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d\": container with ID starting with 797cbf6263858fac6c7666bd9452cbf99a7b893112a201eb74b625230c08a66d not found: ID does not exist"
Sep 10 21:34:54.987443 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987417    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-credential-socket\") pod \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") "
Sep 10 21:34:54.987573 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987455    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-data\") pod \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") "
Sep 10 21:34:54.987573 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987508    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istiod-ca-cert\") pod \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") "
Sep 10 21:34:54.987573 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987567    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-certs\") pod \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") "
Sep 10 21:34:54.987742 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987597    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-socket\") pod \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") "
Sep 10 21:34:54.987742 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987635    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gtt2\" (UniqueName: \"kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-kube-api-access-9gtt2\") pod \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") "
Sep 10 21:34:54.987742 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987670    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6ee4f0c7-4eaa-4753-96e4-911c26186cf7" (UID: "6ee4f0c7-4eaa-4753-96e4-911c26186cf7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:54.987742 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987675    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-token\") pod \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") "
Sep 10 21:34:54.987966 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987760    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-data" (OuterVolumeSpecName: "istio-data") pod "6ee4f0c7-4eaa-4753-96e4-911c26186cf7" (UID: "6ee4f0c7-4eaa-4753-96e4-911c26186cf7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:54.987966 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987829    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-envoy\") pod \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") "
Sep 10 21:34:54.987966 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987863    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-podinfo\") pod \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\" (UID: \"6ee4f0c7-4eaa-4753-96e4-911c26186cf7\") "
Sep 10 21:34:54.987966 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987902    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6ee4f0c7-4eaa-4753-96e4-911c26186cf7" (UID: "6ee4f0c7-4eaa-4753-96e4-911c26186cf7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:54.987966 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.987953    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6ee4f0c7-4eaa-4753-96e4-911c26186cf7" (UID: "6ee4f0c7-4eaa-4753-96e4-911c26186cf7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:54.988219 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.988049    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6ee4f0c7-4eaa-4753-96e4-911c26186cf7" (UID: "6ee4f0c7-4eaa-4753-96e4-911c26186cf7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:54.988219 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.988202    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:54.988219 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.988217    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:54.988378 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.988225    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:54.988378 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.988233    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:54.988378 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.988241    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:54.990015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.989983    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6ee4f0c7-4eaa-4753-96e4-911c26186cf7" (UID: "6ee4f0c7-4eaa-4753-96e4-911c26186cf7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:34:54.990106 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.990062    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6ee4f0c7-4eaa-4753-96e4-911c26186cf7" (UID: "6ee4f0c7-4eaa-4753-96e4-911c26186cf7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:54.990348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.990323    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-kube-api-access-9gtt2" (OuterVolumeSpecName: "kube-api-access-9gtt2") pod "6ee4f0c7-4eaa-4753-96e4-911c26186cf7" (UID: "6ee4f0c7-4eaa-4753-96e4-911c26186cf7"). InnerVolumeSpecName "kube-api-access-9gtt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:54.990454 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:54.990353    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-token" (OuterVolumeSpecName: "istio-token") pod "6ee4f0c7-4eaa-4753-96e4-911c26186cf7" (UID: "6ee4f0c7-4eaa-4753-96e4-911c26186cf7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:55.089474 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:55.089441    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gtt2\" (UniqueName: \"kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-kube-api-access-9gtt2\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:55.089474 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:55.089466    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:55.089474 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:55.089479    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:55.089711 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:55.089491    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ee4f0c7-4eaa-4753-96e4-911c26186cf7-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:34:55.262884 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:55.262853    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"]
Sep 10 21:34:55.266132 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:55.266104    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-c71bd0-openshift-default-55766b75c6-mx5rd"]
Sep 10 21:34:56.679542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:34:56.679510    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" path="/var/lib/kubelet/pods/6ee4f0c7-4eaa-4753-96e4-911c26186cf7/volumes"
Sep 10 21:35:00.518310 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.518272    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"]
Sep 10 21:35:00.518706 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.518573    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9" podUID="934d789c-6bdc-4722-8430-ddc91cc9495d" containerName="maas-api" containerID="cri-o://f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f" gracePeriod=30
Sep 10 21:35:00.554319 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.554282    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs"]
Sep 10 21:35:00.556727 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.556699    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bd7b8a-29817930-s9wqs"]
Sep 10 21:35:00.601094 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.601054    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9" podUID="934d789c-6bdc-4722-8430-ddc91cc9495d" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.64:8443/health\": dial tcp 10.134.0.64:8443: connect: connection refused"
Sep 10 21:35:00.681011 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.680973    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="870622d9-6221-4b78-a5a3-15ea1f941fc8" path="/var/lib/kubelet/pods/870622d9-6221-4b78-a5a3-15ea1f941fc8/volumes"
Sep 10 21:35:00.765785 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.765747    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:35:00.841373 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.841342    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-maas-api-tls\") pod \"934d789c-6bdc-4722-8430-ddc91cc9495d\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") "
Sep 10 21:35:00.841538 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.841392    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/934d789c-6bdc-4722-8430-ddc91cc9495d-openshift-service-ca\") pod \"934d789c-6bdc-4722-8430-ddc91cc9495d\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") "
Sep 10 21:35:00.841538 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.841449    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-metrics-certs\") pod \"934d789c-6bdc-4722-8430-ddc91cc9495d\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") "
Sep 10 21:35:00.841538 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.841490    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2plt9\" (UniqueName: \"kubernetes.io/projected/934d789c-6bdc-4722-8430-ddc91cc9495d-kube-api-access-2plt9\") pod \"934d789c-6bdc-4722-8430-ddc91cc9495d\" (UID: \"934d789c-6bdc-4722-8430-ddc91cc9495d\") "
Sep 10 21:35:00.841771 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.841749    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/934d789c-6bdc-4722-8430-ddc91cc9495d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "934d789c-6bdc-4722-8430-ddc91cc9495d" (UID: "934d789c-6bdc-4722-8430-ddc91cc9495d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:00.843609 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.843579    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "934d789c-6bdc-4722-8430-ddc91cc9495d" (UID: "934d789c-6bdc-4722-8430-ddc91cc9495d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:35:00.843737 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.843610    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "934d789c-6bdc-4722-8430-ddc91cc9495d" (UID: "934d789c-6bdc-4722-8430-ddc91cc9495d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:35:00.843737 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.843636    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934d789c-6bdc-4722-8430-ddc91cc9495d-kube-api-access-2plt9" (OuterVolumeSpecName: "kube-api-access-2plt9") pod "934d789c-6bdc-4722-8430-ddc91cc9495d" (UID: "934d789c-6bdc-4722-8430-ddc91cc9495d"). InnerVolumeSpecName "kube-api-access-2plt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:00.942575 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.942543    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2plt9\" (UniqueName: \"kubernetes.io/projected/934d789c-6bdc-4722-8430-ddc91cc9495d-kube-api-access-2plt9\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:00.942575 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.942574    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-maas-api-tls\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:00.942575 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.942583    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/934d789c-6bdc-4722-8430-ddc91cc9495d-openshift-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:00.942828 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.942592    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/934d789c-6bdc-4722-8430-ddc91cc9495d-metrics-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:00.977099 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.977070    2573 generic.go:356] "Generic (PLEG): container finished" podID="934d789c-6bdc-4722-8430-ddc91cc9495d" containerID="f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f" exitCode=0
Sep 10 21:35:00.977242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.977116    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"
Sep 10 21:35:00.977242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.977157    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9" event={"ID":"934d789c-6bdc-4722-8430-ddc91cc9495d","Type":"ContainerDied","Data":"f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f"}
Sep 10 21:35:00.977242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.977214    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9" event={"ID":"934d789c-6bdc-4722-8430-ddc91cc9495d","Type":"ContainerDied","Data":"88ff1d8ad00d4d4c926dcda05f97a83e7f89659bbb729726b90a1bd435a8dcd0"}
Sep 10 21:35:00.977351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.977243    2573 scope.go:117] "RemoveContainer" containerID="f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f"
Sep 10 21:35:00.987549 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.987530    2573 scope.go:117] "RemoveContainer" containerID="f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f"
Sep 10 21:35:00.987878 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:35:00.987851    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f\": container with ID starting with f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f not found: ID does not exist" containerID="f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f"
Sep 10 21:35:00.987943 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.987891    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f"} err="failed to get container status \"f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f\": rpc error: code = NotFound desc = could not find container \"f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f\": container with ID starting with f5957d420bb579b526eb2f1800c07eea4cc218c4185e9ae77fe47b70ff86631f not found: ID does not exist"
Sep 10 21:35:01.000014 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:00.999980    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"]
Sep 10 21:35:01.005688 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:01.005664    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd7b8a-586b849c66-zq9k9"]
Sep 10 21:35:02.679143 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:02.679117    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="934d789c-6bdc-4722-8430-ddc91cc9495d" path="/var/lib/kubelet/pods/934d789c-6bdc-4722-8430-ddc91cc9495d/volumes"
Sep 10 21:35:05.517443 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:05.517403    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:35:07.202727 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.202693    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"]
Sep 10 21:35:07.203205 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.202949    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy" containerID="cri-o://83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed" gracePeriod=30
Sep 10 21:35:07.594393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594356    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"]
Sep 10 21:35:07.594743 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594731    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy"
Sep 10 21:35:07.594809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594744    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy"
Sep 10 21:35:07.594809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594753    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy"
Sep 10 21:35:07.594809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594759    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy"
Sep 10 21:35:07.594809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594770    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy"
Sep 10 21:35:07.594809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594791    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy"
Sep 10 21:35:07.594809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594805    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5" containerName="revoke-keys"
Sep 10 21:35:07.594809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594810    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5" containerName="revoke-keys"
Sep 10 21:35:07.595037 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594822    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="934d789c-6bdc-4722-8430-ddc91cc9495d" containerName="maas-api"
Sep 10 21:35:07.595037 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594828    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="934d789c-6bdc-4722-8430-ddc91cc9495d" containerName="maas-api"
Sep 10 21:35:07.595037 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594883    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="934d789c-6bdc-4722-8430-ddc91cc9495d" containerName="maas-api"
Sep 10 21:35:07.595037 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594890    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy"
Sep 10 21:35:07.595037 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594899    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bbdc468-9bda-47a4-a2a3-15ce18b1c0c5" containerName="revoke-keys"
Sep 10 21:35:07.595037 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594904    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ee4f0c7-4eaa-4753-96e4-911c26186cf7" containerName="istio-proxy"
Sep 10 21:35:07.595037 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594910    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy"
Sep 10 21:35:07.595037 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.594917    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy"
Sep 10 21:35:07.597943 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.597927    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"
Sep 10 21:35:07.600611 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.600591    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:35:07.600726 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.600686    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:35:07.621635 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.621606    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"]
Sep 10 21:35:07.706929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.706889    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8ca378-00ea-46ac-b3af-5104f9b5c949-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs\" (UID: \"df8ca378-00ea-46ac-b3af-5104f9b5c949\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"
Sep 10 21:35:07.807487 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.807452    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8ca378-00ea-46ac-b3af-5104f9b5c949-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs\" (UID: \"df8ca378-00ea-46ac-b3af-5104f9b5c949\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"
Sep 10 21:35:07.808186 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.808164    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8ca378-00ea-46ac-b3af-5104f9b5c949-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs\" (UID: \"df8ca378-00ea-46ac-b3af-5104f9b5c949\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"
Sep 10 21:35:07.908230 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:07.908135    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"
Sep 10 21:35:08.180491 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:08.180446    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"]
Sep 10 21:35:08.182629 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:35:08.182601    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8ca378_00ea_46ac_b3af_5104f9b5c949.slice/crio-a5b6007206e47deb759d92145c0dba0189514fa92db7b0345ac8e36072f2c0b6 WatchSource:0}: Error finding container a5b6007206e47deb759d92145c0dba0189514fa92db7b0345ac8e36072f2c0b6: Status 404 returned error can't find the container with id a5b6007206e47deb759d92145c0dba0189514fa92db7b0345ac8e36072f2c0b6
Sep 10 21:35:09.025104 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:09.025067    2573 generic.go:356] "Generic (PLEG): container finished" podID="df8ca378-00ea-46ac-b3af-5104f9b5c949" containerID="c9f2ec6ab13c1ec356cec7fba58f72857e1657138fd5455b5ce6f69cac47c5f3" exitCode=0
Sep 10 21:35:09.025528 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:09.025154    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs" event={"ID":"df8ca378-00ea-46ac-b3af-5104f9b5c949","Type":"ContainerDied","Data":"c9f2ec6ab13c1ec356cec7fba58f72857e1657138fd5455b5ce6f69cac47c5f3"}
Sep 10 21:35:09.025528 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:09.025222    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs" event={"ID":"df8ca378-00ea-46ac-b3af-5104f9b5c949","Type":"ContainerStarted","Data":"a5b6007206e47deb759d92145c0dba0189514fa92db7b0345ac8e36072f2c0b6"}
Sep 10 21:35:09.592998 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:09.592957    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"]
Sep 10 21:35:10.171392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:10.171362    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"
Sep 10 21:35:10.330402 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:10.330370    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8ca378-00ea-46ac-b3af-5104f9b5c949-maas-api-service-ca\") pod \"df8ca378-00ea-46ac-b3af-5104f9b5c949\" (UID: \"df8ca378-00ea-46ac-b3af-5104f9b5c949\") "
Sep 10 21:35:10.330753 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:10.330731    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8ca378-00ea-46ac-b3af-5104f9b5c949-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "df8ca378-00ea-46ac-b3af-5104f9b5c949" (UID: "df8ca378-00ea-46ac-b3af-5104f9b5c949"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:10.431540 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:10.431500    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8ca378-00ea-46ac-b3af-5104f9b5c949-maas-api-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:10.731526 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:10.731494    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"]
Sep 10 21:35:11.044541 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:11.044453    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"
Sep 10 21:35:11.044541 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:11.044485    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs" event={"ID":"df8ca378-00ea-46ac-b3af-5104f9b5c949","Type":"ContainerDied","Data":"a5b6007206e47deb759d92145c0dba0189514fa92db7b0345ac8e36072f2c0b6"}
Sep 10 21:35:11.044541 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:11.044536    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5b6007206e47deb759d92145c0dba0189514fa92db7b0345ac8e36072f2c0b6"
Sep 10 21:35:12.456842 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.456819    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:35:12.554872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.554764    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ab16a8-300d-43ee-8d96-454c3f5788fa-istiod-ca-cert\") pod \"85ab16a8-300d-43ee-8d96-454c3f5788fa\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") "
Sep 10 21:35:12.554872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.554822    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-token\") pod \"85ab16a8-300d-43ee-8d96-454c3f5788fa\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") "
Sep 10 21:35:12.555092 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.554882    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-envoy\") pod \"85ab16a8-300d-43ee-8d96-454c3f5788fa\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") "
Sep 10 21:35:12.555092 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.554906    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-podinfo\") pod \"85ab16a8-300d-43ee-8d96-454c3f5788fa\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") "
Sep 10 21:35:12.555092 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.554931    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-credential-socket\") pod \"85ab16a8-300d-43ee-8d96-454c3f5788fa\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") "
Sep 10 21:35:12.555092 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.554963    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-socket\") pod \"85ab16a8-300d-43ee-8d96-454c3f5788fa\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") "
Sep 10 21:35:12.555092 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.554992    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-data\") pod \"85ab16a8-300d-43ee-8d96-454c3f5788fa\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") "
Sep 10 21:35:12.555092 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.555054    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpqn2\" (UniqueName: \"kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-kube-api-access-rpqn2\") pod \"85ab16a8-300d-43ee-8d96-454c3f5788fa\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") "
Sep 10 21:35:12.555412 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.555102    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-certs\") pod \"85ab16a8-300d-43ee-8d96-454c3f5788fa\" (UID: \"85ab16a8-300d-43ee-8d96-454c3f5788fa\") "
Sep 10 21:35:12.555412 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.555231    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85ab16a8-300d-43ee-8d96-454c3f5788fa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85ab16a8-300d-43ee-8d96-454c3f5788fa" (UID: "85ab16a8-300d-43ee-8d96-454c3f5788fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:12.555412 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.555283    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-data" (OuterVolumeSpecName: "istio-data") pod "85ab16a8-300d-43ee-8d96-454c3f5788fa" (UID: "85ab16a8-300d-43ee-8d96-454c3f5788fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:12.555412 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.555307    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85ab16a8-300d-43ee-8d96-454c3f5788fa" (UID: "85ab16a8-300d-43ee-8d96-454c3f5788fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:12.555610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.555439    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85ab16a8-300d-43ee-8d96-454c3f5788fa" (UID: "85ab16a8-300d-43ee-8d96-454c3f5788fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:12.555610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.555497    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ab16a8-300d-43ee-8d96-454c3f5788fa-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:12.555610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.555517    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:12.555610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.555532    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:12.555610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.555544    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85ab16a8-300d-43ee-8d96-454c3f5788fa" (UID: "85ab16a8-300d-43ee-8d96-454c3f5788fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:12.557174 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.557153    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "85ab16a8-300d-43ee-8d96-454c3f5788fa" (UID: "85ab16a8-300d-43ee-8d96-454c3f5788fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:12.557342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.557325    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85ab16a8-300d-43ee-8d96-454c3f5788fa" (UID: "85ab16a8-300d-43ee-8d96-454c3f5788fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:35:12.557588 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.557571    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-kube-api-access-rpqn2" (OuterVolumeSpecName: "kube-api-access-rpqn2") pod "85ab16a8-300d-43ee-8d96-454c3f5788fa" (UID: "85ab16a8-300d-43ee-8d96-454c3f5788fa"). InnerVolumeSpecName "kube-api-access-rpqn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:12.557645 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.557581    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85ab16a8-300d-43ee-8d96-454c3f5788fa" (UID: "85ab16a8-300d-43ee-8d96-454c3f5788fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:12.656900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.656863    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:12.656900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.656892    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:12.656900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.656901    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ab16a8-300d-43ee-8d96-454c3f5788fa-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:12.656900 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.656909    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:12.657180 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.656919    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpqn2\" (UniqueName: \"kubernetes.io/projected/85ab16a8-300d-43ee-8d96-454c3f5788fa-kube-api-access-rpqn2\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:12.657180 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:12.656929    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ab16a8-300d-43ee-8d96-454c3f5788fa-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:13.058399 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.058365    2573 generic.go:356] "Generic (PLEG): container finished" podID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerID="83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed" exitCode=0
Sep 10 21:35:13.058590 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.058426    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"
Sep 10 21:35:13.058590 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.058454    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" event={"ID":"85ab16a8-300d-43ee-8d96-454c3f5788fa","Type":"ContainerDied","Data":"83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed"}
Sep 10 21:35:13.058590 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.058508    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g" event={"ID":"85ab16a8-300d-43ee-8d96-454c3f5788fa","Type":"ContainerDied","Data":"9e8426a5083bcd5ac71de51a55c2c182651e4122d21e1805e8bd06b684a07013"}
Sep 10 21:35:13.058590 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.058531    2573 scope.go:117] "RemoveContainer" containerID="83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed"
Sep 10 21:35:13.067798 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.067765    2573 scope.go:117] "RemoveContainer" containerID="ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc"
Sep 10 21:35:13.076281 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.076262    2573 scope.go:117] "RemoveContainer" containerID="83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed"
Sep 10 21:35:13.076343 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.076309    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"]
Sep 10 21:35:13.076573 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:35:13.076556    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed\": container with ID starting with 83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed not found: ID does not exist" containerID="83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed"
Sep 10 21:35:13.076643 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.076586    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed"} err="failed to get container status \"83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed\": rpc error: code = NotFound desc = could not find container \"83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed\": container with ID starting with 83fa4e70d6f3f95c6cbb75206b888d8558440a680cae18d1f8d20394ae8136ed not found: ID does not exist"
Sep 10 21:35:13.076643 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.076612    2573 scope.go:117] "RemoveContainer" containerID="ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc"
Sep 10 21:35:13.076924 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:35:13.076906    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc\": container with ID starting with ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc not found: ID does not exist" containerID="ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc"
Sep 10 21:35:13.076970 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.076930    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc"} err="failed to get container status \"ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc\": rpc error: code = NotFound desc = could not find container \"ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc\": container with ID starting with ba275364078612e7c5bb259b924384fcf02de5a87bdf18f724899d986d1332dc not found: ID does not exist"
Sep 10 21:35:13.080726 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:13.080704    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bd7b8a-openshift-default-58df5fb486-q2w9g"]
Sep 10 21:35:14.678851 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:14.678814    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" path="/var/lib/kubelet/pods/85ab16a8-300d-43ee-8d96-454c3f5788fa/volumes"
Sep 10 21:35:15.355758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.355720    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"]
Sep 10 21:35:15.356173 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.356159    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy"
Sep 10 21:35:15.356227 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.356175    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy"
Sep 10 21:35:15.356227 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.356184    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy"
Sep 10 21:35:15.356227 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.356190    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed944e7-d060-4ada-9637-048d564bb9c5" containerName="istio-proxy"
Sep 10 21:35:15.356227 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.356206    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy"
Sep 10 21:35:15.356227 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.356212    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy"
Sep 10 21:35:15.356227 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.356219    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8ca378-00ea-46ac-b3af-5104f9b5c949" containerName="revoke-keys"
Sep 10 21:35:15.356227 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.356224    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8ca378-00ea-46ac-b3af-5104f9b5c949" containerName="revoke-keys"
Sep 10 21:35:15.356463 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.356285    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy"
Sep 10 21:35:15.356463 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.356294    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8ca378-00ea-46ac-b3af-5104f9b5c949" containerName="revoke-keys"
Sep 10 21:35:15.359529 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.359513    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.361851 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.361829    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-c8d758-serving-cert\""
Sep 10 21:35:15.362433 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.362410    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:35:15.362566 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.362456    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:35:15.362566 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.362433    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:35:15.362566 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.362433    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-99rzw\""
Sep 10 21:35:15.370456 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.370433    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"]
Sep 10 21:35:15.486869 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.486836    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.487039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.486920    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-metrics-certs\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.487039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.486973    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efea6ace-bd33-463b-b248-e9de595341a0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.487039 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.487035    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xrlw\" (UniqueName: \"kubernetes.io/projected/efea6ace-bd33-463b-b248-e9de595341a0-kube-api-access-8xrlw\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.587992 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.587950    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xrlw\" (UniqueName: \"kubernetes.io/projected/efea6ace-bd33-463b-b248-e9de595341a0-kube-api-access-8xrlw\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.588229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.588023    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.588229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.588087    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-metrics-certs\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.588229 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:35:15.588170    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-serving-cert: secret "maas-api-e2e-ipp-a-c8d758-serving-cert" not found
Sep 10 21:35:15.588229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.588210    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efea6ace-bd33-463b-b248-e9de595341a0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.588399 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:35:15.588242    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-maas-api-tls podName:efea6ace-bd33-463b-b248-e9de595341a0 nodeName:}" failed. No retries permitted until 2026-09-10 21:35:16.088222416 +0000 UTC m=+1492.021889224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-maas-api-tls") pod "maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp" (UID: "efea6ace-bd33-463b-b248-e9de595341a0") : secret "maas-api-e2e-ipp-a-c8d758-serving-cert" not found
Sep 10 21:35:15.588749 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.588732    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efea6ace-bd33-463b-b248-e9de595341a0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.590502 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.590484    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-metrics-certs\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:15.596087 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:15.596061    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xrlw\" (UniqueName: \"kubernetes.io/projected/efea6ace-bd33-463b-b248-e9de595341a0-kube-api-access-8xrlw\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:16.092000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:16.091970    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:16.094467 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:16.094435    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:16.272102 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:16.272063    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:16.544893 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:16.544860    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"]
Sep 10 21:35:16.546082 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:35:16.546057    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefea6ace_bd33_463b_b248_e9de595341a0.slice/crio-3beafb2bdea4a8d04508e47213abd2eb09d21856634e7fd5428a828b88c3c7b5 WatchSource:0}: Error finding container 3beafb2bdea4a8d04508e47213abd2eb09d21856634e7fd5428a828b88c3c7b5: Status 404 returned error can't find the container with id 3beafb2bdea4a8d04508e47213abd2eb09d21856634e7fd5428a828b88c3c7b5
Sep 10 21:35:16.547474 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:16.547456    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:35:17.086595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:17.086554    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp" event={"ID":"efea6ace-bd33-463b-b248-e9de595341a0","Type":"ContainerStarted","Data":"9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13"}
Sep 10 21:35:17.086740 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:17.086617    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp" event={"ID":"efea6ace-bd33-463b-b248-e9de595341a0","Type":"ContainerStarted","Data":"3beafb2bdea4a8d04508e47213abd2eb09d21856634e7fd5428a828b88c3c7b5"}
Sep 10 21:35:17.086740 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:17.086644    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:17.107366 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:17.107312    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp" podStartSLOduration=1.6634751250000002 podStartE2EDuration="2.10729587s" totalImagesPullingTime="443.820745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:17.105472835 +0000 UTC m=+1493.039139649" watchObservedRunningTime="2026-09-10 21:35:17.10729587 +0000 UTC m=+1493.040962685"
Sep 10 21:35:18.646255 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:18.646212    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"]
Sep 10 21:35:18.648599 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:18.648577    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5e6835-jpbrs"]
Sep 10 21:35:18.679561 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:18.679523    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df8ca378-00ea-46ac-b3af-5104f9b5c949" path="/var/lib/kubelet/pods/df8ca378-00ea-46ac-b3af-5104f9b5c949/volumes"
Sep 10 21:35:19.916942 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:19.916901    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:35:23.098610 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:23.098577    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:35:24.608800 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.608754    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:35:24.612249 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.612217    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:35:24.613175 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.613139    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n" podUID="c8632dad-c12a-4af4-98ea-0c0a993ca59e" containerName="main" containerID="cri-o://0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773" gracePeriod=45
Sep 10 21:35:24.871444 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.871417    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:35:24.971393 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.971356    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tls-certs\") pod \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") "
Sep 10 21:35:24.971594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.971407    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-home\") pod \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") "
Sep 10 21:35:24.971594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.971435    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-model-cache\") pod \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") "
Sep 10 21:35:24.971730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.971601    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tmp-dir\") pod \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") "
Sep 10 21:35:24.971730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.971656    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-dshm\") pod \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") "
Sep 10 21:35:24.971730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.971675    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-home" (OuterVolumeSpecName: "home") pod "c8632dad-c12a-4af4-98ea-0c0a993ca59e" (UID: "c8632dad-c12a-4af4-98ea-0c0a993ca59e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:24.971730 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.971690    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-model-cache" (OuterVolumeSpecName: "model-cache") pod "c8632dad-c12a-4af4-98ea-0c0a993ca59e" (UID: "c8632dad-c12a-4af4-98ea-0c0a993ca59e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:24.971938 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.971736    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnt2c\" (UniqueName: \"kubernetes.io/projected/c8632dad-c12a-4af4-98ea-0c0a993ca59e-kube-api-access-qnt2c\") pod \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\" (UID: \"c8632dad-c12a-4af4-98ea-0c0a993ca59e\") "
Sep 10 21:35:24.971938 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.971904    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c8632dad-c12a-4af4-98ea-0c0a993ca59e" (UID: "c8632dad-c12a-4af4-98ea-0c0a993ca59e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:24.972144 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.972114    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:24.972144 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.972139    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:24.972277 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.972150    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:24.973676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.973657    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-dshm" (OuterVolumeSpecName: "dshm") pod "c8632dad-c12a-4af4-98ea-0c0a993ca59e" (UID: "c8632dad-c12a-4af4-98ea-0c0a993ca59e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:24.973830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.973750    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c8632dad-c12a-4af4-98ea-0c0a993ca59e" (UID: "c8632dad-c12a-4af4-98ea-0c0a993ca59e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:35:24.973830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:24.973797    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8632dad-c12a-4af4-98ea-0c0a993ca59e-kube-api-access-qnt2c" (OuterVolumeSpecName: "kube-api-access-qnt2c") pod "c8632dad-c12a-4af4-98ea-0c0a993ca59e" (UID: "c8632dad-c12a-4af4-98ea-0c0a993ca59e"). InnerVolumeSpecName "kube-api-access-qnt2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:25.073215 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.073176    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnt2c\" (UniqueName: \"kubernetes.io/projected/c8632dad-c12a-4af4-98ea-0c0a993ca59e-kube-api-access-qnt2c\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:25.073215 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.073211    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8632dad-c12a-4af4-98ea-0c0a993ca59e-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:25.073215 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.073223    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8632dad-c12a-4af4-98ea-0c0a993ca59e-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:25.132594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.132499    2573 generic.go:356] "Generic (PLEG): container finished" podID="c8632dad-c12a-4af4-98ea-0c0a993ca59e" containerID="0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773" exitCode=0
Sep 10 21:35:25.132594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.132551    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"
Sep 10 21:35:25.132594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.132582    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n" event={"ID":"c8632dad-c12a-4af4-98ea-0c0a993ca59e","Type":"ContainerDied","Data":"0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773"}
Sep 10 21:35:25.132879 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.132632    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n" event={"ID":"c8632dad-c12a-4af4-98ea-0c0a993ca59e","Type":"ContainerDied","Data":"cd4014b095fe628ad9f9d53a8a19e863c4d72e0531ec29ff997a706e9b56f3d3"}
Sep 10 21:35:25.132879 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.132655    2573 scope.go:117] "RemoveContainer" containerID="0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773"
Sep 10 21:35:25.142865 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.142845    2573 scope.go:117] "RemoveContainer" containerID="0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773"
Sep 10 21:35:25.143117 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:35:25.143101    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773\": container with ID starting with 0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773 not found: ID does not exist" containerID="0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773"
Sep 10 21:35:25.143166 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.143124    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773"} err="failed to get container status \"0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773\": rpc error: code = NotFound desc = could not find container \"0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773\": container with ID starting with 0cdfd5949d8ecea4c0e404f867e70e52f42cf311ab4147db5f95187d91667773 not found: ID does not exist"
Sep 10 21:35:25.154363 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.154340    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"]
Sep 10 21:35:25.158166 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.158141    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-36b938/rate-test-model-36b938-kserve-6f6bb69595-5pr2n"]
Sep 10 21:35:25.414826 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.414702    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"]
Sep 10 21:35:25.415044 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.415017    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy" containerID="cri-o://5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d" gracePeriod=30
Sep 10 21:35:25.750725 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:25.750639    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks" podUID="7fff0526-6efe-4634-8b19-04b2e036a44c" containerName="main" containerID="cri-o://fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263" gracePeriod=45
Sep 10 21:35:26.007736 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.007665    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:35:26.081506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.081471    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fff0526-6efe-4634-8b19-04b2e036a44c-tls-certs\") pod \"7fff0526-6efe-4634-8b19-04b2e036a44c\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") "
Sep 10 21:35:26.081506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.081512    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j2wc\" (UniqueName: \"kubernetes.io/projected/7fff0526-6efe-4634-8b19-04b2e036a44c-kube-api-access-5j2wc\") pod \"7fff0526-6efe-4634-8b19-04b2e036a44c\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") "
Sep 10 21:35:26.081716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.081574    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-dshm\") pod \"7fff0526-6efe-4634-8b19-04b2e036a44c\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") "
Sep 10 21:35:26.081716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.081596    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-home\") pod \"7fff0526-6efe-4634-8b19-04b2e036a44c\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") "
Sep 10 21:35:26.081716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.081646    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-tmp-dir\") pod \"7fff0526-6efe-4634-8b19-04b2e036a44c\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") "
Sep 10 21:35:26.081716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.081713    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-model-cache\") pod \"7fff0526-6efe-4634-8b19-04b2e036a44c\" (UID: \"7fff0526-6efe-4634-8b19-04b2e036a44c\") "
Sep 10 21:35:26.082008 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.081981    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-home" (OuterVolumeSpecName: "home") pod "7fff0526-6efe-4634-8b19-04b2e036a44c" (UID: "7fff0526-6efe-4634-8b19-04b2e036a44c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:26.082077 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.082006    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7fff0526-6efe-4634-8b19-04b2e036a44c" (UID: "7fff0526-6efe-4634-8b19-04b2e036a44c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:26.082077 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.082002    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-model-cache" (OuterVolumeSpecName: "model-cache") pod "7fff0526-6efe-4634-8b19-04b2e036a44c" (UID: "7fff0526-6efe-4634-8b19-04b2e036a44c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:26.082188 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.082075    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:26.082188 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.082090    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:26.083878 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.083848    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fff0526-6efe-4634-8b19-04b2e036a44c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7fff0526-6efe-4634-8b19-04b2e036a44c" (UID: "7fff0526-6efe-4634-8b19-04b2e036a44c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:35:26.084000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.083887    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fff0526-6efe-4634-8b19-04b2e036a44c-kube-api-access-5j2wc" (OuterVolumeSpecName: "kube-api-access-5j2wc") pod "7fff0526-6efe-4634-8b19-04b2e036a44c" (UID: "7fff0526-6efe-4634-8b19-04b2e036a44c"). InnerVolumeSpecName "kube-api-access-5j2wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:26.084000 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.083898    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-dshm" (OuterVolumeSpecName: "dshm") pod "7fff0526-6efe-4634-8b19-04b2e036a44c" (UID: "7fff0526-6efe-4634-8b19-04b2e036a44c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:26.140519 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.140488    2573 generic.go:356] "Generic (PLEG): container finished" podID="7fff0526-6efe-4634-8b19-04b2e036a44c" containerID="fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263" exitCode=0
Sep 10 21:35:26.140682 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.140532    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"
Sep 10 21:35:26.140682 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.140571    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks" event={"ID":"7fff0526-6efe-4634-8b19-04b2e036a44c","Type":"ContainerDied","Data":"fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263"}
Sep 10 21:35:26.140682 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.140616    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks" event={"ID":"7fff0526-6efe-4634-8b19-04b2e036a44c","Type":"ContainerDied","Data":"97640b5a30f05abbf6e5e19179f9de80c95012da3412355b87f9b6377e4340c6"}
Sep 10 21:35:26.140682 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.140634    2573 scope.go:117] "RemoveContainer" containerID="fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263"
Sep 10 21:35:26.150311 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.150294    2573 scope.go:117] "RemoveContainer" containerID="fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263"
Sep 10 21:35:26.150593 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:35:26.150574    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263\": container with ID starting with fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263 not found: ID does not exist" containerID="fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263"
Sep 10 21:35:26.150647 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.150603    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263"} err="failed to get container status \"fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263\": rpc error: code = NotFound desc = could not find container \"fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263\": container with ID starting with fea43a1f183930f1e865066b04862066a82745f541fd6c3f1c5312ea9d5b3263 not found: ID does not exist"
Sep 10 21:35:26.162348 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.162320    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"]
Sep 10 21:35:26.167635 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.167612    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1f3879/rate-test-model-1f3879-kserve-5c8649459f-qqvks"]
Sep 10 21:35:26.182767 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.182744    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:26.182767 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.182766    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fff0526-6efe-4634-8b19-04b2e036a44c-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:26.182911 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.182792    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j2wc\" (UniqueName: \"kubernetes.io/projected/7fff0526-6efe-4634-8b19-04b2e036a44c-kube-api-access-5j2wc\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:26.182911 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.182803    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fff0526-6efe-4634-8b19-04b2e036a44c-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:26.678846 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.678816    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fff0526-6efe-4634-8b19-04b2e036a44c" path="/var/lib/kubelet/pods/7fff0526-6efe-4634-8b19-04b2e036a44c/volumes"
Sep 10 21:35:26.679248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:26.679236    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8632dad-c12a-4af4-98ea-0c0a993ca59e" path="/var/lib/kubelet/pods/c8632dad-c12a-4af4-98ea-0c0a993ca59e/volumes"
Sep 10 21:35:28.405683 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:28.405648    2573 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:35:28.406228 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:28.405722    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:35:30.665294 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.665268    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:35:30.726595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.726503    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istiod-ca-cert\") pod \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") "
Sep 10 21:35:30.726595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.726542    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-envoy\") pod \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") "
Sep 10 21:35:30.726595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.726584    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-credential-socket\") pod \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") "
Sep 10 21:35:30.726851 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.726766    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-token\") pod \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") "
Sep 10 21:35:30.726899 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.726881    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkblp\" (UniqueName: \"kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-kube-api-access-pkblp\") pod \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") "
Sep 10 21:35:30.726944 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.726919    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "96b9a14d-d6cb-405b-bbd1-cce61559fbb4" (UID: "96b9a14d-d6cb-405b-bbd1-cce61559fbb4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:30.726944 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.726929    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "96b9a14d-d6cb-405b-bbd1-cce61559fbb4" (UID: "96b9a14d-d6cb-405b-bbd1-cce61559fbb4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:30.726944 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.726939    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-certs\") pod \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") "
Sep 10 21:35:30.727100 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.726973    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-data\") pod \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") "
Sep 10 21:35:30.727100 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.726999    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-socket\") pod \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") "
Sep 10 21:35:30.727100 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.727062    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-podinfo\") pod \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\" (UID: \"96b9a14d-d6cb-405b-bbd1-cce61559fbb4\") "
Sep 10 21:35:30.727262 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.727216    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-data" (OuterVolumeSpecName: "istio-data") pod "96b9a14d-d6cb-405b-bbd1-cce61559fbb4" (UID: "96b9a14d-d6cb-405b-bbd1-cce61559fbb4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:30.727262 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.727248    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "96b9a14d-d6cb-405b-bbd1-cce61559fbb4" (UID: "96b9a14d-d6cb-405b-bbd1-cce61559fbb4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:30.727347 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.727287    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "96b9a14d-d6cb-405b-bbd1-cce61559fbb4" (UID: "96b9a14d-d6cb-405b-bbd1-cce61559fbb4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:30.727432 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.727409    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:30.727432 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.727433    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:30.727599 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.727447    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:30.727599 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.727460    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:30.727599 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.727474    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:30.729208 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.729186    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "96b9a14d-d6cb-405b-bbd1-cce61559fbb4" (UID: "96b9a14d-d6cb-405b-bbd1-cce61559fbb4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:30.729304 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.729265    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-token" (OuterVolumeSpecName: "istio-token") pod "96b9a14d-d6cb-405b-bbd1-cce61559fbb4" (UID: "96b9a14d-d6cb-405b-bbd1-cce61559fbb4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:30.729531 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.729513    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "96b9a14d-d6cb-405b-bbd1-cce61559fbb4" (UID: "96b9a14d-d6cb-405b-bbd1-cce61559fbb4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:35:30.729581 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.729540    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-kube-api-access-pkblp" (OuterVolumeSpecName: "kube-api-access-pkblp") pod "96b9a14d-d6cb-405b-bbd1-cce61559fbb4" (UID: "96b9a14d-d6cb-405b-bbd1-cce61559fbb4"). InnerVolumeSpecName "kube-api-access-pkblp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:30.828887 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.828846    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:30.828887 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.828882    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:30.828887 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.828894    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:30.829141 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:30.828905    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkblp\" (UniqueName: \"kubernetes.io/projected/96b9a14d-d6cb-405b-bbd1-cce61559fbb4-kube-api-access-pkblp\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:35:31.170105 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.170067    2573 generic.go:356] "Generic (PLEG): container finished" podID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerID="5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d" exitCode=0
Sep 10 21:35:31.170307 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.170116    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" event={"ID":"96b9a14d-d6cb-405b-bbd1-cce61559fbb4","Type":"ContainerDied","Data":"5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d"}
Sep 10 21:35:31.170307 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.170134    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"
Sep 10 21:35:31.170307 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.170184    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt" event={"ID":"96b9a14d-d6cb-405b-bbd1-cce61559fbb4","Type":"ContainerDied","Data":"5e70ef920e44d140b2c9c79d6aa3bbe64a13f2184ea57aa3144f3f3668456e9c"}
Sep 10 21:35:31.170307 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.170206    2573 scope.go:117] "RemoveContainer" containerID="5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d"
Sep 10 21:35:31.180126 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.180104    2573 scope.go:117] "RemoveContainer" containerID="7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c"
Sep 10 21:35:31.188653 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.188633    2573 scope.go:117] "RemoveContainer" containerID="5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d"
Sep 10 21:35:31.188924 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:35:31.188908    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d\": container with ID starting with 5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d not found: ID does not exist" containerID="5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d"
Sep 10 21:35:31.188989 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.188939    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d"} err="failed to get container status \"5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d\": rpc error: code = NotFound desc = could not find container \"5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d\": container with ID starting with 5d36588b311003339c3c292488a0366c774b8fcdd172249843d5187fb0f2ca2d not found: ID does not exist"
Sep 10 21:35:31.188989 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.188963    2573 scope.go:117] "RemoveContainer" containerID="7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c"
Sep 10 21:35:31.189212 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:35:31.189192    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c\": container with ID starting with 7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c not found: ID does not exist" containerID="7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c"
Sep 10 21:35:31.189250 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.189218    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c"} err="failed to get container status \"7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c\": rpc error: code = NotFound desc = could not find container \"7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c\": container with ID starting with 7fcd5d2540ad04ba2d51415c3d2d540915262198d06c57b3d7e6bbdcc8e1fe2c not found: ID does not exist"
Sep 10 21:35:31.193406 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.193382    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"]
Sep 10 21:35:31.201785 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:31.201753    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e6835-openshift-default-569764c6cb-p89nt"]
Sep 10 21:35:32.679064 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:32.679032    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" path="/var/lib/kubelet/pods/96b9a14d-d6cb-405b-bbd1-cce61559fbb4/volumes"
Sep 10 21:35:34.014356 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:34.014323    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:35:40.166211 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166175    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"]
Sep 10 21:35:40.166592 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166576    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fff0526-6efe-4634-8b19-04b2e036a44c" containerName="main"
Sep 10 21:35:40.166640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166594    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fff0526-6efe-4634-8b19-04b2e036a44c" containerName="main"
Sep 10 21:35:40.166640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166608    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8632dad-c12a-4af4-98ea-0c0a993ca59e" containerName="main"
Sep 10 21:35:40.166640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166614    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8632dad-c12a-4af4-98ea-0c0a993ca59e" containerName="main"
Sep 10 21:35:40.166640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166623    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy"
Sep 10 21:35:40.166640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166629    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy"
Sep 10 21:35:40.166818 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166642    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy"
Sep 10 21:35:40.166818 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166647    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy"
Sep 10 21:35:40.166818 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166712    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ab16a8-300d-43ee-8d96-454c3f5788fa" containerName="istio-proxy"
Sep 10 21:35:40.166818 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166720    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8632dad-c12a-4af4-98ea-0c0a993ca59e" containerName="main"
Sep 10 21:35:40.166818 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166726    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy"
Sep 10 21:35:40.166818 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166732    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fff0526-6efe-4634-8b19-04b2e036a44c" containerName="main"
Sep 10 21:35:40.166818 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.166739    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="96b9a14d-d6cb-405b-bbd1-cce61559fbb4" containerName="istio-proxy"
Sep 10 21:35:40.170409 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.170387    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.173159 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.173138    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-5f25ef-openshift-default-dockercfg-ztmmz\""
Sep 10 21:35:40.194724 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.194702    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"]
Sep 10 21:35:40.212856 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.212830    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-credential-socket\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.212966 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.212867    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-istio-token\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.212966 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.212905    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-envoy\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.213045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.212961    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-socket\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.213045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.213023    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-data\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.213117 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.213050    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1418d7f5-81a7-406c-a204-f5afbe440779-istiod-ca-cert\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.213117 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.213071    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-certs\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.213117 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.213094    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfsnq\" (UniqueName: \"kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-kube-api-access-kfsnq\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.213231 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.213126    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1418d7f5-81a7-406c-a204-f5afbe440779-istio-podinfo\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314011 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.313975    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-envoy\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314011 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314014    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-socket\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314046    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-data\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314072    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1418d7f5-81a7-406c-a204-f5afbe440779-istiod-ca-cert\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314099    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-certs\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314133    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfsnq\" (UniqueName: \"kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-kube-api-access-kfsnq\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314188    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1418d7f5-81a7-406c-a204-f5afbe440779-istio-podinfo\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314251    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-credential-socket\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314286 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314280    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-istio-token\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314649 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314537    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-socket\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314649 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314608    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-certs\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314822 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314797    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-credential-socket\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314944 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314854    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-data\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.314989 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.314961    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1418d7f5-81a7-406c-a204-f5afbe440779-istiod-ca-cert\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.316997 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.316969    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-envoy\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.317222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.317201    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1418d7f5-81a7-406c-a204-f5afbe440779-istio-podinfo\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.326755 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.326726    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfsnq\" (UniqueName: \"kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-kube-api-access-kfsnq\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.326868 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.326766    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-istio-token\") pod \"e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") " pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.480754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.480668    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:40.758253 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.758221    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"]
Sep 10 21:35:40.760185 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:35:40.760163    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1418d7f5_81a7_406c_a204_f5afbe440779.slice/crio-77c6f26abc3868bf931aa7a638ca602300d1e7c62e1df650bb277d8419499529 WatchSource:0}: Error finding container 77c6f26abc3868bf931aa7a638ca602300d1e7c62e1df650bb277d8419499529: Status 404 returned error can't find the container with id 77c6f26abc3868bf931aa7a638ca602300d1e7c62e1df650bb277d8419499529
Sep 10 21:35:40.762289 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.762259    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:35:40.762383 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.762320    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:35:40.762383 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:40.762347    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:35:41.229933 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:41.229899    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" event={"ID":"1418d7f5-81a7-406c-a204-f5afbe440779","Type":"ContainerStarted","Data":"71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096"}
Sep 10 21:35:41.230325 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:41.229949    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" event={"ID":"1418d7f5-81a7-406c-a204-f5afbe440779","Type":"ContainerStarted","Data":"77c6f26abc3868bf931aa7a638ca602300d1e7c62e1df650bb277d8419499529"}
Sep 10 21:35:41.283449 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:41.283389    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" podStartSLOduration=1.2833687409999999 podStartE2EDuration="1.283368741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:35:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:41.283309162 +0000 UTC m=+1517.216975979" watchObservedRunningTime="2026-09-10 21:35:41.283368741 +0000 UTC m=+1517.217035557"
Sep 10 21:35:41.481131 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:41.481029    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:41.486125 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:41.486098    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:42.236200 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:42.236165    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:42.237252 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:42.237231    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:35:44.369529 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.369488    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"]
Sep 10 21:35:44.372194 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.372173    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.374805 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.374765    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-5f25ef-serving-cert\""
Sep 10 21:35:44.385256 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.385231    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"]
Sep 10 21:35:44.455536 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.455506    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca01c627-4be1-4235-9ab6-c37ea718bdc5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.455698 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.455542    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzgb6\" (UniqueName: \"kubernetes.io/projected/ca01c627-4be1-4235-9ab6-c37ea718bdc5-kube-api-access-zzgb6\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.455750 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.455690    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.455810 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.455752    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-metrics-certs\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.557211 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.557171    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.557412 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.557229    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-metrics-certs\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.557412 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.557274    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca01c627-4be1-4235-9ab6-c37ea718bdc5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.557412 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.557302    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzgb6\" (UniqueName: \"kubernetes.io/projected/ca01c627-4be1-4235-9ab6-c37ea718bdc5-kube-api-access-zzgb6\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.558005 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.557982    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca01c627-4be1-4235-9ab6-c37ea718bdc5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.559829 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.559807    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-metrics-certs\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.559910 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.559878    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.565029 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.565010    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzgb6\" (UniqueName: \"kubernetes.io/projected/ca01c627-4be1-4235-9ab6-c37ea718bdc5-kube-api-access-zzgb6\") pod \"maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.684755 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.684679    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:44.966947 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:44.966922    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"]
Sep 10 21:35:44.968718 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:35:44.968686    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca01c627_4be1_4235_9ab6_c37ea718bdc5.slice/crio-545ec8878de3f39f378acf6ed214fb1c0d202c076c692bda71015554a9cbd127 WatchSource:0}: Error finding container 545ec8878de3f39f378acf6ed214fb1c0d202c076c692bda71015554a9cbd127: Status 404 returned error can't find the container with id 545ec8878de3f39f378acf6ed214fb1c0d202c076c692bda71015554a9cbd127
Sep 10 21:35:45.260351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:45.260274    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5" event={"ID":"ca01c627-4be1-4235-9ab6-c37ea718bdc5","Type":"ContainerStarted","Data":"545ec8878de3f39f378acf6ed214fb1c0d202c076c692bda71015554a9cbd127"}
Sep 10 21:35:46.267977 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:46.267940    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5" event={"ID":"ca01c627-4be1-4235-9ab6-c37ea718bdc5","Type":"ContainerStarted","Data":"6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0"}
Sep 10 21:35:46.268375 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:46.268006    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:46.285368 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:46.285322    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5" podStartSLOduration=1.982823732 podStartE2EDuration="2.285306141s" totalImagesPullingTime="302.482409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:46.284745593 +0000 UTC m=+1522.218412409" watchObservedRunningTime="2026-09-10 21:35:46.285306141 +0000 UTC m=+1522.218972984"
Sep 10 21:35:48.817888 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:48.817850    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:35:52.285140 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:52.285087    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:35:58.278021 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:58.277963    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 21:35:58.278556 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:58.278073    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 21:35:59.352727 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:59.352696    2573 generic.go:356] "Generic (PLEG): container finished" podID="1418d7f5-81a7-406c-a204-f5afbe440779" containerID="71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096" exitCode=0
Sep 10 21:35:59.353167 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:59.352758    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" event={"ID":"1418d7f5-81a7-406c-a204-f5afbe440779","Type":"ContainerDied","Data":"71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096"}
Sep 10 21:35:59.353242 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:59.353225    2573 scope.go:117] "RemoveContainer" containerID="71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096"
Sep 10 21:35:59.354354 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:59.354313    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:35:59.354462 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:59.354409    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:35:59.354534 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:35:59.354461    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:36:00.361074 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:00.361040    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" event={"ID":"1418d7f5-81a7-406c-a204-f5afbe440779","Type":"ContainerStarted","Data":"60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b"}
Sep 10 21:36:00.481366 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:00.481330    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:36:00.481661 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:00.481393    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Sep 10 21:36:00.481872 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:00.481846    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Sep 10 21:36:01.481630 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:01.481577    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Sep 10 21:36:01.482179 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:01.481652    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Sep 10 21:36:02.482213 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:02.482161    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Sep 10 21:36:02.482750 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:02.482243    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Sep 10 21:36:03.481486 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:03.481434    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Sep 10 21:36:03.481734 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:03.481527    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Sep 10 21:36:04.482141 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:04.482099    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Sep 10 21:36:04.482676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:04.482190    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Sep 10 21:36:05.487192 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:05.487154    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:36:05.487729 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:05.487701    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:36:05.488438 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:05.488418    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:36:17.013564 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.013522    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"]
Sep 10 21:36:17.017186 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.017158    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.019555 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.019529    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-c8d758\"/\"kube-root-ca.crt\""
Sep 10 21:36:17.020232 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.020208    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-c8d758\"/\"default-dockercfg-7nxdh\""
Sep 10 21:36:17.020353 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.020219    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-c8d758\"/\"ipp-route-c8d758-kserve-self-signed-certs\""
Sep 10 21:36:17.020353 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.020267    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-c8d758\"/\"openshift-service-ca.crt\""
Sep 10 21:36:17.029569 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.029546    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"]
Sep 10 21:36:17.060854 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.060820    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq7cc\" (UniqueName: \"kubernetes.io/projected/9566faf4-f223-4fe6-ab38-696be0c687b0-kube-api-access-zq7cc\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.061038 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.060879    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9566faf4-f223-4fe6-ab38-696be0c687b0-tls-certs\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.061038 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.060917    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-model-cache\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.061038 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.060941    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-dshm\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.061038 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.060958    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-home\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.061182 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.061036    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-tmp-dir\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.162175 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.162131    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-dshm\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.162175 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.162175    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-home\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.162431 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.162229    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-tmp-dir\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.162431 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.162259    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq7cc\" (UniqueName: \"kubernetes.io/projected/9566faf4-f223-4fe6-ab38-696be0c687b0-kube-api-access-zq7cc\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.162431 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.162311    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9566faf4-f223-4fe6-ab38-696be0c687b0-tls-certs\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.162431 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.162343    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-model-cache\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.162692 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.162667    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-tmp-dir\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.162754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.162696    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-home\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.162754 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.162710    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-model-cache\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.164595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.164574    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-dshm\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.164718 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.164703    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9566faf4-f223-4fe6-ab38-696be0c687b0-tls-certs\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.171143 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.171122    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq7cc\" (UniqueName: \"kubernetes.io/projected/9566faf4-f223-4fe6-ab38-696be0c687b0-kube-api-access-zq7cc\") pod \"ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") " pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.327542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.327510    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:17.599831 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:17.599796    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"]
Sep 10 21:36:17.600890 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:36:17.600864    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9566faf4_f223_4fe6_ab38_696be0c687b0.slice/crio-ac2c48ecf762acf5ea9c51d5cf0b5a35d11e8b9d637e8be416bf93cb1ccbcabb WatchSource:0}: Error finding container ac2c48ecf762acf5ea9c51d5cf0b5a35d11e8b9d637e8be416bf93cb1ccbcabb: Status 404 returned error can't find the container with id ac2c48ecf762acf5ea9c51d5cf0b5a35d11e8b9d637e8be416bf93cb1ccbcabb
Sep 10 21:36:18.476851 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:18.476807    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6" event={"ID":"9566faf4-f223-4fe6-ab38-696be0c687b0","Type":"ContainerStarted","Data":"3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d"}
Sep 10 21:36:18.476851 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:18.476854    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6" event={"ID":"9566faf4-f223-4fe6-ab38-696be0c687b0","Type":"ContainerStarted","Data":"ac2c48ecf762acf5ea9c51d5cf0b5a35d11e8b9d637e8be416bf93cb1ccbcabb"}
Sep 10 21:36:18.494290 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:18.494121    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6" podStartSLOduration=2.494103215 podStartE2EDuration="2.494103215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:36:18.492933249 +0000 UTC m=+1554.426600063" watchObservedRunningTime="2026-09-10 21:36:18.494103215 +0000 UTC m=+1554.427770031"
Sep 10 21:36:20.710124 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:20.710086    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:36:24.873149 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:24.873113    2573 scope.go:117] "RemoveContainer" containerID="99e21933839eb450b5423f73d11be5367d8f98e117b4dc04d6a61c4d0d541405"
Sep 10 21:36:27.327743 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:27.327695    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:27.328233 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:27.327761    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:27.340605 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:27.340576    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:27.539341 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:27.539307    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:36:33.922482 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:33.922442    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:36:41.593341 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.593306    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"]
Sep 10 21:36:41.598280 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.598260    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.601123 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.601096    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-389568-openshift-default-dockercfg-6w626\""
Sep 10 21:36:41.617283 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.617257    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"]
Sep 10 21:36:41.708600 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.708562    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-token\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.708841 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.708609    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-certs\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.708841 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.708647    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-envoy\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.708841 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.708741    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-data\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.708841 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.708820    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istiod-ca-cert\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.709049 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.708847    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdwsb\" (UniqueName: \"kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-kube-api-access-jdwsb\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.709049 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.708892    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-socket\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.709049 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.708950    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-credential-socket\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.709049 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.708977    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-podinfo\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.809927 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.809891    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-credential-socket\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.809927 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.809928    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-podinfo\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810187 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.809965    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-token\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810187 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.809996    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-certs\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810187 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.810037    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-envoy\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810187 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.810104    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-data\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810187 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.810153    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istiod-ca-cert\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810187 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.810175    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdwsb\" (UniqueName: \"kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-kube-api-access-jdwsb\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.810231    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-socket\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.810362    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-credential-socket\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.810403    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-certs\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810801 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.810752    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-socket\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.810992 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.810969    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-data\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.811164 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.811148    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istiod-ca-cert\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.812529 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.812507    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-podinfo\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.812529 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.812517    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-envoy\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.817811 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.817760    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-token\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.817942 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.817926    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdwsb\" (UniqueName: \"kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-kube-api-access-jdwsb\") pod \"e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") " pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:41.909705 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:41.909603    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:42.190750 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:42.190723    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"]
Sep 10 21:36:42.192584 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:36:42.192554    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08b88cf9_16e4_4bd8_9afe_6a4394c00b15.slice/crio-dd9c356f9a7d77686a094a6548ba2bd1a338ce50838326def7c902f08fdcb6c2 WatchSource:0}: Error finding container dd9c356f9a7d77686a094a6548ba2bd1a338ce50838326def7c902f08fdcb6c2: Status 404 returned error can't find the container with id dd9c356f9a7d77686a094a6548ba2bd1a338ce50838326def7c902f08fdcb6c2
Sep 10 21:36:42.195472 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:42.195432    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:36:42.195542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:42.195519    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:36:42.195581 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:42.195566    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:36:42.614634 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:42.614594    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" event={"ID":"08b88cf9-16e4-4bd8-9afe-6a4394c00b15","Type":"ContainerStarted","Data":"6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c"}
Sep 10 21:36:42.615108 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:42.614656    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" event={"ID":"08b88cf9-16e4-4bd8-9afe-6a4394c00b15","Type":"ContainerStarted","Data":"dd9c356f9a7d77686a094a6548ba2bd1a338ce50838326def7c902f08fdcb6c2"}
Sep 10 21:36:42.631658 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:42.631609    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" podStartSLOduration=1.631593536 podStartE2EDuration="1.631593536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:36:42.631489411 +0000 UTC m=+1578.565156226" watchObservedRunningTime="2026-09-10 21:36:42.631593536 +0000 UTC m=+1578.565260349"
Sep 10 21:36:42.910807 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:42.910688    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:43.915430 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:43.915396    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:44.626933 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:44.626902    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:44.627817 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:44.627764    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:36:52.906244 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:36:52.906206    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:37:03.752187 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:03.752155    2573 generic.go:356] "Generic (PLEG): container finished" podID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerID="6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c" exitCode=0
Sep 10 21:37:03.752623 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:03.752234    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" event={"ID":"08b88cf9-16e4-4bd8-9afe-6a4394c00b15","Type":"ContainerDied","Data":"6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c"}
Sep 10 21:37:03.752675 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:03.752641    2573 scope.go:117] "RemoveContainer" containerID="6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c"
Sep 10 21:37:03.756630 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:03.754322    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:37:03.756630 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:03.754507    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:37:03.756630 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:03.754605    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 10 21:37:04.760058 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:04.760020    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" event={"ID":"08b88cf9-16e4-4bd8-9afe-6a4394c00b15","Type":"ContainerStarted","Data":"fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e"}
Sep 10 21:37:04.909980 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:04.909939    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:37:04.910205 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:04.910182    2573 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 10 21:37:04.910255 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:04.910232    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 10 21:37:05.910701 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:05.910662    2573 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 10 21:37:05.911112 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:05.910732    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 10 21:37:06.911053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:06.911015    2573 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 10 21:37:06.911457 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:06.911073    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 10 21:37:07.910713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:07.910676    2573 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 10 21:37:07.910925 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:07.910731    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 10 21:37:08.910096 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:08.910058    2573 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 10 21:37:08.910502 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:08.910120    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 10 21:37:09.914415 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:09.914379    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:37:09.914809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:09.914742    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:37:09.915498 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:09.915481    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:37:20.712239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:20.712158    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:37:22.806769 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:22.806732    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"]
Sep 10 21:37:22.807629 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:22.807594    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy" containerID="cri-o://fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e" gracePeriod=30
Sep 10 21:37:24.916969 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:24.916930    2573 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:37:24.917403 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:24.917002    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:37:28.050152 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.050127    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:37:28.077086 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077061    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-certs\") pod \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") "
Sep 10 21:37:28.077245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077109    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-envoy\") pod \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") "
Sep 10 21:37:28.077245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077135    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdwsb\" (UniqueName: \"kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-kube-api-access-jdwsb\") pod \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") "
Sep 10 21:37:28.077245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077151    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-socket\") pod \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") "
Sep 10 21:37:28.077245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077186    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-podinfo\") pod \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") "
Sep 10 21:37:28.077245 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077221    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-credential-socket\") pod \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") "
Sep 10 21:37:28.077519 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077385    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "08b88cf9-16e4-4bd8-9afe-6a4394c00b15" (UID: "08b88cf9-16e4-4bd8-9afe-6a4394c00b15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:37:28.077519 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077440    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-data\") pod \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") "
Sep 10 21:37:28.077627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077517    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "08b88cf9-16e4-4bd8-9afe-6a4394c00b15" (UID: "08b88cf9-16e4-4bd8-9afe-6a4394c00b15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:37:28.077627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077530    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "08b88cf9-16e4-4bd8-9afe-6a4394c00b15" (UID: "08b88cf9-16e4-4bd8-9afe-6a4394c00b15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:37:28.077627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077538    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istiod-ca-cert\") pod \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") "
Sep 10 21:37:28.077627 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077606    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-token\") pod \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\" (UID: \"08b88cf9-16e4-4bd8-9afe-6a4394c00b15\") "
Sep 10 21:37:28.077838 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077707    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-data" (OuterVolumeSpecName: "istio-data") pod "08b88cf9-16e4-4bd8-9afe-6a4394c00b15" (UID: "08b88cf9-16e4-4bd8-9afe-6a4394c00b15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:37:28.077890 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.077848    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "08b88cf9-16e4-4bd8-9afe-6a4394c00b15" (UID: "08b88cf9-16e4-4bd8-9afe-6a4394c00b15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:28.078065 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.078043    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:28.078065 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.078067    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:28.078253 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.078083    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:28.078253 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.078094    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:28.078253 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.078103    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:28.079571 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.079543    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "08b88cf9-16e4-4bd8-9afe-6a4394c00b15" (UID: "08b88cf9-16e4-4bd8-9afe-6a4394c00b15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:37:28.079933 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.079907    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "08b88cf9-16e4-4bd8-9afe-6a4394c00b15" (UID: "08b88cf9-16e4-4bd8-9afe-6a4394c00b15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:37:28.080045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.079937    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-kube-api-access-jdwsb" (OuterVolumeSpecName: "kube-api-access-jdwsb") pod "08b88cf9-16e4-4bd8-9afe-6a4394c00b15" (UID: "08b88cf9-16e4-4bd8-9afe-6a4394c00b15"). InnerVolumeSpecName "kube-api-access-jdwsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:28.080045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.079960    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-token" (OuterVolumeSpecName: "istio-token") pod "08b88cf9-16e4-4bd8-9afe-6a4394c00b15" (UID: "08b88cf9-16e4-4bd8-9afe-6a4394c00b15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:28.179306 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.179273    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:28.179306 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.179302    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdwsb\" (UniqueName: \"kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-kube-api-access-jdwsb\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:28.179306 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.179312    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:28.179544 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.179321    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08b88cf9-16e4-4bd8-9afe-6a4394c00b15-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:28.907233 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.907200    2573 generic.go:356] "Generic (PLEG): container finished" podID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerID="fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e" exitCode=0
Sep 10 21:37:28.907425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.907259    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"
Sep 10 21:37:28.907425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.907289    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" event={"ID":"08b88cf9-16e4-4bd8-9afe-6a4394c00b15","Type":"ContainerDied","Data":"fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e"}
Sep 10 21:37:28.907425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.907347    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s" event={"ID":"08b88cf9-16e4-4bd8-9afe-6a4394c00b15","Type":"ContainerDied","Data":"dd9c356f9a7d77686a094a6548ba2bd1a338ce50838326def7c902f08fdcb6c2"}
Sep 10 21:37:28.907425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.907379    2573 scope.go:117] "RemoveContainer" containerID="fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e"
Sep 10 21:37:28.917003 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.916988    2573 scope.go:117] "RemoveContainer" containerID="6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c"
Sep 10 21:37:28.925488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.925302    2573 scope.go:117] "RemoveContainer" containerID="fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e"
Sep 10 21:37:28.925605 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:37:28.925583    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e\": container with ID starting with fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e not found: ID does not exist" containerID="fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e"
Sep 10 21:37:28.925646 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.925612    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e"} err="failed to get container status \"fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e\": rpc error: code = NotFound desc = could not find container \"fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e\": container with ID starting with fa03db892bfc616c7d4913d43f47eb1b9eb46066c0731e3aeffd68f9a925726e not found: ID does not exist"
Sep 10 21:37:28.925646 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.925631    2573 scope.go:117] "RemoveContainer" containerID="6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c"
Sep 10 21:37:28.925878 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:37:28.925859    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c\": container with ID starting with 6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c not found: ID does not exist" containerID="6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c"
Sep 10 21:37:28.925946 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.925888    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c"} err="failed to get container status \"6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c\": rpc error: code = NotFound desc = could not find container \"6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c\": container with ID starting with 6c1a979b36717506594f04f9a4944a885f84258f5309c9a14d4c712dd8e0a30c not found: ID does not exist"
Sep 10 21:37:28.938831 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.938764    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"]
Sep 10 21:37:28.942875 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:28.942856    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389568-openshift-default-755c865474-ddp4s"]
Sep 10 21:37:30.679672 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:30.679639    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" path="/var/lib/kubelet/pods/08b88cf9-16e4-4bd8-9afe-6a4394c00b15/volumes"
Sep 10 21:37:39.714889 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:39.714856    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:37:41.943845 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:41.943809    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"]
Sep 10 21:37:41.944269 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:41.944063    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp" podUID="efea6ace-bd33-463b-b248-e9de595341a0" containerName="maas-api" containerID="cri-o://9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13" gracePeriod=30
Sep 10 21:37:42.196413 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.196358    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:37:42.304304 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.304272    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efea6ace-bd33-463b-b248-e9de595341a0-openshift-service-ca\") pod \"efea6ace-bd33-463b-b248-e9de595341a0\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") "
Sep 10 21:37:42.304465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.304321    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-metrics-certs\") pod \"efea6ace-bd33-463b-b248-e9de595341a0\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") "
Sep 10 21:37:42.304465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.304368    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xrlw\" (UniqueName: \"kubernetes.io/projected/efea6ace-bd33-463b-b248-e9de595341a0-kube-api-access-8xrlw\") pod \"efea6ace-bd33-463b-b248-e9de595341a0\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") "
Sep 10 21:37:42.304465 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.304454    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-maas-api-tls\") pod \"efea6ace-bd33-463b-b248-e9de595341a0\" (UID: \"efea6ace-bd33-463b-b248-e9de595341a0\") "
Sep 10 21:37:42.304713 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.304689    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efea6ace-bd33-463b-b248-e9de595341a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efea6ace-bd33-463b-b248-e9de595341a0" (UID: "efea6ace-bd33-463b-b248-e9de595341a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:42.304824 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.304806    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efea6ace-bd33-463b-b248-e9de595341a0-openshift-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:42.306676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.306642    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efea6ace-bd33-463b-b248-e9de595341a0" (UID: "efea6ace-bd33-463b-b248-e9de595341a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:37:42.306813 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.306683    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "efea6ace-bd33-463b-b248-e9de595341a0" (UID: "efea6ace-bd33-463b-b248-e9de595341a0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:37:42.306813 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.306720    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efea6ace-bd33-463b-b248-e9de595341a0-kube-api-access-8xrlw" (OuterVolumeSpecName: "kube-api-access-8xrlw") pod "efea6ace-bd33-463b-b248-e9de595341a0" (UID: "efea6ace-bd33-463b-b248-e9de595341a0"). InnerVolumeSpecName "kube-api-access-8xrlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:42.405595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.405551    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-metrics-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:42.405595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.405579    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xrlw\" (UniqueName: \"kubernetes.io/projected/efea6ace-bd33-463b-b248-e9de595341a0-kube-api-access-8xrlw\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:42.405595 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.405589    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea6ace-bd33-463b-b248-e9de595341a0-maas-api-tls\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:37:42.991239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.991151    2573 generic.go:356] "Generic (PLEG): container finished" podID="efea6ace-bd33-463b-b248-e9de595341a0" containerID="9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13" exitCode=0
Sep 10 21:37:42.991239 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.991200    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"
Sep 10 21:37:42.991659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.991240    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp" event={"ID":"efea6ace-bd33-463b-b248-e9de595341a0","Type":"ContainerDied","Data":"9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13"}
Sep 10 21:37:42.991659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.991296    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp" event={"ID":"efea6ace-bd33-463b-b248-e9de595341a0","Type":"ContainerDied","Data":"3beafb2bdea4a8d04508e47213abd2eb09d21856634e7fd5428a828b88c3c7b5"}
Sep 10 21:37:42.991659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:42.991318    2573 scope.go:117] "RemoveContainer" containerID="9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13"
Sep 10 21:37:43.000676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:43.000660    2573 scope.go:117] "RemoveContainer" containerID="9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13"
Sep 10 21:37:43.000926 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:37:43.000909    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13\": container with ID starting with 9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13 not found: ID does not exist" containerID="9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13"
Sep 10 21:37:43.000976 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:43.000934    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13"} err="failed to get container status \"9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13\": rpc error: code = NotFound desc = could not find container \"9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13\": container with ID starting with 9127c517b91a22e7e309d61164c27c7f4485c7b100bec3b4467d2c374ca42d13 not found: ID does not exist"
Sep 10 21:37:43.010129 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:43.010105    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"]
Sep 10 21:37:43.016556 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:43.016536    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c8d758-74bccbf6f-8ddwp"]
Sep 10 21:37:44.679483 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:44.679451    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efea6ace-bd33-463b-b248-e9de595341a0" path="/var/lib/kubelet/pods/efea6ace-bd33-463b-b248-e9de595341a0/volumes"
Sep 10 21:37:47.518493 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:47.518457    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"]
Sep 10 21:37:52.209091 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:37:52.209053    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:38:02.537511 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.537446    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6" podUID="9566faf4-f223-4fe6-ab38-696be0c687b0" containerName="main" containerID="cri-o://3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d" gracePeriod=45
Sep 10 21:38:02.778144 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.778123    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:38:02.879904 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.879875    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-tmp-dir\") pod \"9566faf4-f223-4fe6-ab38-696be0c687b0\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") "
Sep 10 21:38:02.880077 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.879929    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq7cc\" (UniqueName: \"kubernetes.io/projected/9566faf4-f223-4fe6-ab38-696be0c687b0-kube-api-access-zq7cc\") pod \"9566faf4-f223-4fe6-ab38-696be0c687b0\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") "
Sep 10 21:38:02.880077 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.879978    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-model-cache\") pod \"9566faf4-f223-4fe6-ab38-696be0c687b0\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") "
Sep 10 21:38:02.880077 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.879995    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-dshm\") pod \"9566faf4-f223-4fe6-ab38-696be0c687b0\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") "
Sep 10 21:38:02.880206 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.880087    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-home\") pod \"9566faf4-f223-4fe6-ab38-696be0c687b0\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") "
Sep 10 21:38:02.880206 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.880134    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9566faf4-f223-4fe6-ab38-696be0c687b0-tls-certs\") pod \"9566faf4-f223-4fe6-ab38-696be0c687b0\" (UID: \"9566faf4-f223-4fe6-ab38-696be0c687b0\") "
Sep 10 21:38:02.880294 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.880192    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9566faf4-f223-4fe6-ab38-696be0c687b0" (UID: "9566faf4-f223-4fe6-ab38-696be0c687b0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:02.880342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.880287    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-model-cache" (OuterVolumeSpecName: "model-cache") pod "9566faf4-f223-4fe6-ab38-696be0c687b0" (UID: "9566faf4-f223-4fe6-ab38-696be0c687b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:02.880413 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.880385    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-home" (OuterVolumeSpecName: "home") pod "9566faf4-f223-4fe6-ab38-696be0c687b0" (UID: "9566faf4-f223-4fe6-ab38-696be0c687b0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:02.880484 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.880460    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:02.880538 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.880489    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:02.882195 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.882171    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9566faf4-f223-4fe6-ab38-696be0c687b0-kube-api-access-zq7cc" (OuterVolumeSpecName: "kube-api-access-zq7cc") pod "9566faf4-f223-4fe6-ab38-696be0c687b0" (UID: "9566faf4-f223-4fe6-ab38-696be0c687b0"). InnerVolumeSpecName "kube-api-access-zq7cc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:02.882301 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.882215    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-dshm" (OuterVolumeSpecName: "dshm") pod "9566faf4-f223-4fe6-ab38-696be0c687b0" (UID: "9566faf4-f223-4fe6-ab38-696be0c687b0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:02.882343 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.882300    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9566faf4-f223-4fe6-ab38-696be0c687b0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9566faf4-f223-4fe6-ab38-696be0c687b0" (UID: "9566faf4-f223-4fe6-ab38-696be0c687b0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:38:02.981455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.981418    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq7cc\" (UniqueName: \"kubernetes.io/projected/9566faf4-f223-4fe6-ab38-696be0c687b0-kube-api-access-zq7cc\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:02.981455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.981449    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:02.981654 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.981463    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9566faf4-f223-4fe6-ab38-696be0c687b0-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:02.981654 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:02.981474    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9566faf4-f223-4fe6-ab38-696be0c687b0-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:03.103654 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:03.103621    2573 generic.go:356] "Generic (PLEG): container finished" podID="9566faf4-f223-4fe6-ab38-696be0c687b0" containerID="3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d" exitCode=0
Sep 10 21:38:03.103822 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:03.103661    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"
Sep 10 21:38:03.103822 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:03.103674    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6" event={"ID":"9566faf4-f223-4fe6-ab38-696be0c687b0","Type":"ContainerDied","Data":"3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d"}
Sep 10 21:38:03.103822 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:03.103715    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6" event={"ID":"9566faf4-f223-4fe6-ab38-696be0c687b0","Type":"ContainerDied","Data":"ac2c48ecf762acf5ea9c51d5cf0b5a35d11e8b9d637e8be416bf93cb1ccbcabb"}
Sep 10 21:38:03.103822 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:03.103737    2573 scope.go:117] "RemoveContainer" containerID="3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d"
Sep 10 21:38:03.113618 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:03.113602    2573 scope.go:117] "RemoveContainer" containerID="3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d"
Sep 10 21:38:03.113875 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:38:03.113854    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d\": container with ID starting with 3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d not found: ID does not exist" containerID="3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d"
Sep 10 21:38:03.113931 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:03.113885    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d"} err="failed to get container status \"3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d\": rpc error: code = NotFound desc = could not find container \"3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d\": container with ID starting with 3467efe5801351f775da56048f01aeb09af0ba76aadd55059b8e78b9ed7db57d not found: ID does not exist"
Sep 10 21:38:03.125259 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:03.125238    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"]
Sep 10 21:38:03.127475 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:03.127452    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-c8d758/ipp-route-c8d758-kserve-5fb6d4f86c-9gvk6"]
Sep 10 21:38:04.679211 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:04.679181    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9566faf4-f223-4fe6-ab38-696be0c687b0" path="/var/lib/kubelet/pods/9566faf4-f223-4fe6-ab38-696be0c687b0/volumes"
Sep 10 21:38:41.234070 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.234032    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"]
Sep 10 21:38:41.234634 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.234310    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5" podUID="ca01c627-4be1-4235-9ab6-c37ea718bdc5" containerName="maas-api" containerID="cri-o://6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0" gracePeriod=30
Sep 10 21:38:41.480123 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.480099    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:38:41.619762 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.619733    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-maas-api-tls\") pod \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") "
Sep 10 21:38:41.619961 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.619887    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca01c627-4be1-4235-9ab6-c37ea718bdc5-openshift-service-ca\") pod \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") "
Sep 10 21:38:41.619961 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.619908    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzgb6\" (UniqueName: \"kubernetes.io/projected/ca01c627-4be1-4235-9ab6-c37ea718bdc5-kube-api-access-zzgb6\") pod \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") "
Sep 10 21:38:41.619961 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.619927    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-metrics-certs\") pod \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\" (UID: \"ca01c627-4be1-4235-9ab6-c37ea718bdc5\") "
Sep 10 21:38:41.620340 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.620311    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca01c627-4be1-4235-9ab6-c37ea718bdc5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca01c627-4be1-4235-9ab6-c37ea718bdc5" (UID: "ca01c627-4be1-4235-9ab6-c37ea718bdc5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:38:41.622203 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.622170    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ca01c627-4be1-4235-9ab6-c37ea718bdc5" (UID: "ca01c627-4be1-4235-9ab6-c37ea718bdc5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:38:41.622203 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.622181    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca01c627-4be1-4235-9ab6-c37ea718bdc5" (UID: "ca01c627-4be1-4235-9ab6-c37ea718bdc5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:38:41.622587 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.622563    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca01c627-4be1-4235-9ab6-c37ea718bdc5-kube-api-access-zzgb6" (OuterVolumeSpecName: "kube-api-access-zzgb6") pod "ca01c627-4be1-4235-9ab6-c37ea718bdc5" (UID: "ca01c627-4be1-4235-9ab6-c37ea718bdc5"). InnerVolumeSpecName "kube-api-access-zzgb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:41.721105 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.721069    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca01c627-4be1-4235-9ab6-c37ea718bdc5-openshift-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:41.721105 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.721109    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzgb6\" (UniqueName: \"kubernetes.io/projected/ca01c627-4be1-4235-9ab6-c37ea718bdc5-kube-api-access-zzgb6\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:41.721325 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.721123    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-metrics-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:41.721325 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:41.721136    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca01c627-4be1-4235-9ab6-c37ea718bdc5-maas-api-tls\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:42.326893 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:42.326856    2573 generic.go:356] "Generic (PLEG): container finished" podID="ca01c627-4be1-4235-9ab6-c37ea718bdc5" containerID="6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0" exitCode=0
Sep 10 21:38:42.327283 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:42.326904    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"
Sep 10 21:38:42.327283 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:42.326943    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5" event={"ID":"ca01c627-4be1-4235-9ab6-c37ea718bdc5","Type":"ContainerDied","Data":"6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0"}
Sep 10 21:38:42.327283 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:42.326997    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5" event={"ID":"ca01c627-4be1-4235-9ab6-c37ea718bdc5","Type":"ContainerDied","Data":"545ec8878de3f39f378acf6ed214fb1c0d202c076c692bda71015554a9cbd127"}
Sep 10 21:38:42.327283 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:42.327020    2573 scope.go:117] "RemoveContainer" containerID="6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0"
Sep 10 21:38:42.338512 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:42.338493    2573 scope.go:117] "RemoveContainer" containerID="6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0"
Sep 10 21:38:42.338827 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:38:42.338803    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0\": container with ID starting with 6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0 not found: ID does not exist" containerID="6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0"
Sep 10 21:38:42.338895 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:42.338842    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0"} err="failed to get container status \"6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0\": rpc error: code = NotFound desc = could not find container \"6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0\": container with ID starting with 6a00b77dc5409870122a62c460f132ad304dc317218198a8a6089eeb33724af0 not found: ID does not exist"
Sep 10 21:38:42.352188 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:42.352163    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"]
Sep 10 21:38:42.355325 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:42.355299    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5f25ef-74b475dbd6-9wns5"]
Sep 10 21:38:42.680106 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:42.680024    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca01c627-4be1-4235-9ab6-c37ea718bdc5" path="/var/lib/kubelet/pods/ca01c627-4be1-4235-9ab6-c37ea718bdc5/volumes"
Sep 10 21:38:44.210518 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:44.210483    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:38:47.840488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:47.840404    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"]
Sep 10 21:38:47.840976 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:47.840640    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy" containerID="cri-o://60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b" gracePeriod=30
Sep 10 21:38:48.192163 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192081    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"]
Sep 10 21:38:48.192441 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192429    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9566faf4-f223-4fe6-ab38-696be0c687b0" containerName="main"
Sep 10 21:38:48.192496 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192442    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9566faf4-f223-4fe6-ab38-696be0c687b0" containerName="main"
Sep 10 21:38:48.192496 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192455    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy"
Sep 10 21:38:48.192496 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192460    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy"
Sep 10 21:38:48.192496 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192480    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efea6ace-bd33-463b-b248-e9de595341a0" containerName="maas-api"
Sep 10 21:38:48.192496 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192486    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="efea6ace-bd33-463b-b248-e9de595341a0" containerName="maas-api"
Sep 10 21:38:48.192496 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192495    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca01c627-4be1-4235-9ab6-c37ea718bdc5" containerName="maas-api"
Sep 10 21:38:48.192708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192500    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca01c627-4be1-4235-9ab6-c37ea718bdc5" containerName="maas-api"
Sep 10 21:38:48.192708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192551    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="efea6ace-bd33-463b-b248-e9de595341a0" containerName="maas-api"
Sep 10 21:38:48.192708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192559    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9566faf4-f223-4fe6-ab38-696be0c687b0" containerName="main"
Sep 10 21:38:48.192708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192565    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy"
Sep 10 21:38:48.192708 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.192572    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca01c627-4be1-4235-9ab6-c37ea718bdc5" containerName="maas-api"
Sep 10 21:38:48.195402 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.195386    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"
Sep 10 21:38:48.197389 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.197361    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:38:48.197531 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.197361    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:38:48.202695 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.202659    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"]
Sep 10 21:38:48.285182 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.285147    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm\" (UID: \"d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"
Sep 10 21:38:48.386073 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.386033    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm\" (UID: \"d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"
Sep 10 21:38:48.386664 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.386642    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm\" (UID: \"d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"
Sep 10 21:38:48.507672 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.507590    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"
Sep 10 21:38:48.774835 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:48.774808    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"]
Sep 10 21:38:48.777647 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:38:48.777612    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e7aa6c_be44_4436_9a6f_f0f1f3f296bd.slice/crio-e95f0081c07df231922dad4f12dc621e7311aa9699df925e4928898bbb3c92ae WatchSource:0}: Error finding container e95f0081c07df231922dad4f12dc621e7311aa9699df925e4928898bbb3c92ae: Status 404 returned error can't find the container with id e95f0081c07df231922dad4f12dc621e7311aa9699df925e4928898bbb3c92ae
Sep 10 21:38:49.382299 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:49.382067    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm" event={"ID":"d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd","Type":"ContainerStarted","Data":"2ad76c40d55a0ee581af637c8c7505bb2a804e523ea425ff3a6d31fc220f94cd"}
Sep 10 21:38:49.382653 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:49.382319    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm" event={"ID":"d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd","Type":"ContainerStarted","Data":"e95f0081c07df231922dad4f12dc621e7311aa9699df925e4928898bbb3c92ae"}
Sep 10 21:38:49.397771 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:49.397717    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm" podStartSLOduration=1.397705199 podStartE2EDuration="1.397705199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:38:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:38:49.395672056 +0000 UTC m=+1705.329338870" watchObservedRunningTime="2026-09-10 21:38:49.397705199 +0000 UTC m=+1705.331372052"
Sep 10 21:38:50.389840 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:50.389804    2573 generic.go:356] "Generic (PLEG): container finished" podID="d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd" containerID="2ad76c40d55a0ee581af637c8c7505bb2a804e523ea425ff3a6d31fc220f94cd" exitCode=0
Sep 10 21:38:50.390241 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:50.389882    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm" event={"ID":"d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd","Type":"ContainerDied","Data":"2ad76c40d55a0ee581af637c8c7505bb2a804e523ea425ff3a6d31fc220f94cd"}
Sep 10 21:38:50.489579 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:50.489545    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:38:50.489734 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:50.489615    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:38:51.520814 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:51.520770    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"
Sep 10 21:38:51.613955 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:51.613923    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd-maas-api-service-ca\") pod \"d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd\" (UID: \"d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd\") "
Sep 10 21:38:51.614341 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:51.614321    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd" (UID: "d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:38:51.715579 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:51.715489    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd-maas-api-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:52.401487 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:52.401455    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"
Sep 10 21:38:52.401667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:52.401505    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm" event={"ID":"d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd","Type":"ContainerDied","Data":"e95f0081c07df231922dad4f12dc621e7311aa9699df925e4928898bbb3c92ae"}
Sep 10 21:38:52.401667 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:52.401550    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e95f0081c07df231922dad4f12dc621e7311aa9699df925e4928898bbb3c92ae"
Sep 10 21:38:53.093029 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.093004    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:38:53.228015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.227912    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-credential-socket\") pod \"1418d7f5-81a7-406c-a204-f5afbe440779\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") "
Sep 10 21:38:53.228015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.227979    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfsnq\" (UniqueName: \"kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-kube-api-access-kfsnq\") pod \"1418d7f5-81a7-406c-a204-f5afbe440779\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") "
Sep 10 21:38:53.228015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228007    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1418d7f5-81a7-406c-a204-f5afbe440779-istio-podinfo\") pod \"1418d7f5-81a7-406c-a204-f5afbe440779\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") "
Sep 10 21:38:53.228015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228023    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-istio-token\") pod \"1418d7f5-81a7-406c-a204-f5afbe440779\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") "
Sep 10 21:38:53.228342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228068    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1418d7f5-81a7-406c-a204-f5afbe440779-istiod-ca-cert\") pod \"1418d7f5-81a7-406c-a204-f5afbe440779\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") "
Sep 10 21:38:53.228342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228092    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-socket\") pod \"1418d7f5-81a7-406c-a204-f5afbe440779\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") "
Sep 10 21:38:53.228342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228108    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-envoy\") pod \"1418d7f5-81a7-406c-a204-f5afbe440779\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") "
Sep 10 21:38:53.228342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228143    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-data\") pod \"1418d7f5-81a7-406c-a204-f5afbe440779\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") "
Sep 10 21:38:53.228342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228197    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-certs\") pod \"1418d7f5-81a7-406c-a204-f5afbe440779\" (UID: \"1418d7f5-81a7-406c-a204-f5afbe440779\") "
Sep 10 21:38:53.228342 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228244    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1418d7f5-81a7-406c-a204-f5afbe440779" (UID: "1418d7f5-81a7-406c-a204-f5afbe440779"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:53.228664 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228478    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-data" (OuterVolumeSpecName: "istio-data") pod "1418d7f5-81a7-406c-a204-f5afbe440779" (UID: "1418d7f5-81a7-406c-a204-f5afbe440779"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:53.228664 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228491    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:53.228664 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228612    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1418d7f5-81a7-406c-a204-f5afbe440779" (UID: "1418d7f5-81a7-406c-a204-f5afbe440779"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:53.228664 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228626    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1418d7f5-81a7-406c-a204-f5afbe440779-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1418d7f5-81a7-406c-a204-f5afbe440779" (UID: "1418d7f5-81a7-406c-a204-f5afbe440779"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:38:53.228831 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.228702    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1418d7f5-81a7-406c-a204-f5afbe440779" (UID: "1418d7f5-81a7-406c-a204-f5afbe440779"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:53.230322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.230299    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1418d7f5-81a7-406c-a204-f5afbe440779" (UID: "1418d7f5-81a7-406c-a204-f5afbe440779"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:53.230435 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.230388    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1418d7f5-81a7-406c-a204-f5afbe440779-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1418d7f5-81a7-406c-a204-f5afbe440779" (UID: "1418d7f5-81a7-406c-a204-f5afbe440779"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:38:53.230697 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.230676    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-kube-api-access-kfsnq" (OuterVolumeSpecName: "kube-api-access-kfsnq") pod "1418d7f5-81a7-406c-a204-f5afbe440779" (UID: "1418d7f5-81a7-406c-a204-f5afbe440779"). InnerVolumeSpecName "kube-api-access-kfsnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:53.230841 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.230706    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-istio-token" (OuterVolumeSpecName: "istio-token") pod "1418d7f5-81a7-406c-a204-f5afbe440779" (UID: "1418d7f5-81a7-406c-a204-f5afbe440779"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:53.329366 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.329326    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfsnq\" (UniqueName: \"kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-kube-api-access-kfsnq\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:53.329366 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.329359    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1418d7f5-81a7-406c-a204-f5afbe440779-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:53.329366 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.329371    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1418d7f5-81a7-406c-a204-f5afbe440779-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:53.329366 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.329378    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1418d7f5-81a7-406c-a204-f5afbe440779-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:53.329640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.329386    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:53.329640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.329394    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:53.329640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.329401    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:53.329640 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.329410    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1418d7f5-81a7-406c-a204-f5afbe440779-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:38:53.408172 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.408137    2573 generic.go:356] "Generic (PLEG): container finished" podID="1418d7f5-81a7-406c-a204-f5afbe440779" containerID="60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b" exitCode=0
Sep 10 21:38:53.408340 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.408193    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"
Sep 10 21:38:53.408340 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.408225    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" event={"ID":"1418d7f5-81a7-406c-a204-f5afbe440779","Type":"ContainerDied","Data":"60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b"}
Sep 10 21:38:53.408340 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.408282    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr" event={"ID":"1418d7f5-81a7-406c-a204-f5afbe440779","Type":"ContainerDied","Data":"77c6f26abc3868bf931aa7a638ca602300d1e7c62e1df650bb277d8419499529"}
Sep 10 21:38:53.408340 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.408304    2573 scope.go:117] "RemoveContainer" containerID="60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b"
Sep 10 21:38:53.418078 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.418057    2573 scope.go:117] "RemoveContainer" containerID="71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096"
Sep 10 21:38:53.426300 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.426279    2573 scope.go:117] "RemoveContainer" containerID="60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b"
Sep 10 21:38:53.426569 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:38:53.426542    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b\": container with ID starting with 60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b not found: ID does not exist" containerID="60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b"
Sep 10 21:38:53.426660 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.426571    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b"} err="failed to get container status \"60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b\": rpc error: code = NotFound desc = could not find container \"60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b\": container with ID starting with 60ad605c4ff84edc493d2be39ae8a21a659b203302cbbc7ec3fa163469ee795b not found: ID does not exist"
Sep 10 21:38:53.426660 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.426589    2573 scope.go:117] "RemoveContainer" containerID="71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096"
Sep 10 21:38:53.426938 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:38:53.426919    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096\": container with ID starting with 71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096 not found: ID does not exist" containerID="71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096"
Sep 10 21:38:53.426993 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.426945    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096"} err="failed to get container status \"71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096\": rpc error: code = NotFound desc = could not find container \"71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096\": container with ID starting with 71d9e2244a5d12cacd90efc0cd7aebc31ea327fdf1366191dceb47cafb537096 not found: ID does not exist"
Sep 10 21:38:53.429878 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.429855    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"]
Sep 10 21:38:53.435479 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:53.435458    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-5f25ef-openshift-default-6c6d5bfb4b-x6pjr"]
Sep 10 21:38:54.679362 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:54.679330    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" path="/var/lib/kubelet/pods/1418d7f5-81a7-406c-a204-f5afbe440779/volumes"
Sep 10 21:38:58.893686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:58.893651    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"]
Sep 10 21:38:58.903056 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:38:58.903029    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-36b938-5l6dm"]
Sep 10 21:39:00.678340 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:00.678309    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd" path="/var/lib/kubelet/pods/d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd/volumes"
Sep 10 21:39:01.609987 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:01.609952    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:39:05.611182 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611145    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"]
Sep 10 21:39:05.611604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611518    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy"
Sep 10 21:39:05.611604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611531    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy"
Sep 10 21:39:05.611604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611547    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy"
Sep 10 21:39:05.611604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611552    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy"
Sep 10 21:39:05.611604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611564    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy"
Sep 10 21:39:05.611604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611572    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy"
Sep 10 21:39:05.611604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611586    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd" containerName="revoke-keys"
Sep 10 21:39:05.611604 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611591    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd" containerName="revoke-keys"
Sep 10 21:39:05.611884 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611647    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b88cf9-16e4-4bd8-9afe-6a4394c00b15" containerName="istio-proxy"
Sep 10 21:39:05.611884 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611658    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy"
Sep 10 21:39:05.611884 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.611666    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e7aa6c-be44-4436-9a6f-f0f1f3f296bd" containerName="revoke-keys"
Sep 10 21:39:05.613953 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.613937    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"
Sep 10 21:39:05.616019 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.615996    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:39:05.616146 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.615998    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:39:05.622720 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.622697    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"]
Sep 10 21:39:05.624208 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.624179    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/521c1595-02fa-4f48-8054-aee7cc84ae01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth\" (UID: \"521c1595-02fa-4f48-8054-aee7cc84ae01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"
Sep 10 21:39:05.725503 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.725462    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/521c1595-02fa-4f48-8054-aee7cc84ae01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth\" (UID: \"521c1595-02fa-4f48-8054-aee7cc84ae01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"
Sep 10 21:39:05.726155 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.726134    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/521c1595-02fa-4f48-8054-aee7cc84ae01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth\" (UID: \"521c1595-02fa-4f48-8054-aee7cc84ae01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"
Sep 10 21:39:05.925170 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:05.925086    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"
Sep 10 21:39:06.195332 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:06.195304    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"]
Sep 10 21:39:06.197029 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:39:06.196998    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521c1595_02fa_4f48_8054_aee7cc84ae01.slice/crio-3f5679512ce06fd4ed628bad9e003353614f28d68adf79832c62fd8390fa74f7 WatchSource:0}: Error finding container 3f5679512ce06fd4ed628bad9e003353614f28d68adf79832c62fd8390fa74f7: Status 404 returned error can't find the container with id 3f5679512ce06fd4ed628bad9e003353614f28d68adf79832c62fd8390fa74f7
Sep 10 21:39:06.489932 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:06.489900    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth" event={"ID":"521c1595-02fa-4f48-8054-aee7cc84ae01","Type":"ContainerStarted","Data":"521e85dc16ffd0fa3c12f8855cc09601188a2ac7c7caa013d3443529246dd745"}
Sep 10 21:39:06.490055 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:06.489970    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth" event={"ID":"521c1595-02fa-4f48-8054-aee7cc84ae01","Type":"ContainerStarted","Data":"3f5679512ce06fd4ed628bad9e003353614f28d68adf79832c62fd8390fa74f7"}
Sep 10 21:39:06.505501 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:06.505431    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth" podStartSLOduration=1.505414898 podStartE2EDuration="1.505414898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:39:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:39:06.502771371 +0000 UTC m=+1722.436438184" watchObservedRunningTime="2026-09-10 21:39:06.505414898 +0000 UTC m=+1722.439081713"
Sep 10 21:39:07.497842 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:07.497809    2573 generic.go:356] "Generic (PLEG): container finished" podID="521c1595-02fa-4f48-8054-aee7cc84ae01" containerID="521e85dc16ffd0fa3c12f8855cc09601188a2ac7c7caa013d3443529246dd745" exitCode=0
Sep 10 21:39:07.498248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:07.497855    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth" event={"ID":"521c1595-02fa-4f48-8054-aee7cc84ae01","Type":"ContainerDied","Data":"521e85dc16ffd0fa3c12f8855cc09601188a2ac7c7caa013d3443529246dd745"}
Sep 10 21:39:08.629461 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:08.629434    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"
Sep 10 21:39:08.646702 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:08.646676    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/521c1595-02fa-4f48-8054-aee7cc84ae01-maas-api-service-ca\") pod \"521c1595-02fa-4f48-8054-aee7cc84ae01\" (UID: \"521c1595-02fa-4f48-8054-aee7cc84ae01\") "
Sep 10 21:39:08.647061 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:08.647040    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/521c1595-02fa-4f48-8054-aee7cc84ae01-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "521c1595-02fa-4f48-8054-aee7cc84ae01" (UID: "521c1595-02fa-4f48-8054-aee7cc84ae01"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:39:08.748146 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:08.747949    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/521c1595-02fa-4f48-8054-aee7cc84ae01-maas-api-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:39:09.516608 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:09.516566    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth" event={"ID":"521c1595-02fa-4f48-8054-aee7cc84ae01","Type":"ContainerDied","Data":"3f5679512ce06fd4ed628bad9e003353614f28d68adf79832c62fd8390fa74f7"}
Sep 10 21:39:09.516608 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:09.516594    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"
Sep 10 21:39:09.516905 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:09.516656    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f5679512ce06fd4ed628bad9e003353614f28d68adf79832c62fd8390fa74f7"
Sep 10 21:39:16.162015 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:16.161979    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk"]
Sep 10 21:39:16.165876 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:16.165843    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1f3879-29817930-h2jkk"]
Sep 10 21:39:16.353895 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:16.353859    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"]
Sep 10 21:39:16.359108 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:16.359082    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1f3879-8xxth"]
Sep 10 21:39:16.685248 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:16.685212    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="521c1595-02fa-4f48-8054-aee7cc84ae01" path="/var/lib/kubelet/pods/521c1595-02fa-4f48-8054-aee7cc84ae01/volumes"
Sep 10 21:39:16.685506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:16.685493    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcfcd359-3cef-47a0-986c-552d7af095f5" path="/var/lib/kubelet/pods/bcfcd359-3cef-47a0-986c-552d7af095f5/volumes"
Sep 10 21:39:18.809223 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:18.809186    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:39:22.693101 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:22.693067    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"]
Sep 10 21:39:22.693567 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:22.693280    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy" containerID="cri-o://f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1" gracePeriod=30
Sep 10 21:39:24.938793 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:24.938753    2573 scope.go:117] "RemoveContainer" containerID="bbf04d1541699d482a3245258ddb8990cbdcf94375ea1979b9608275d0cfff5a"
Sep 10 21:39:26.711310 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:26.711279    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:39:27.948964 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:27.948934    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:39:28.016996 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.016899    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-credential-socket\") pod \"3599639b-5d4d-482f-b00f-24ec11e146e0\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") "
Sep 10 21:39:28.016996 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.016946    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-token\") pod \"3599639b-5d4d-482f-b00f-24ec11e146e0\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") "
Sep 10 21:39:28.016996 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.016979    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-socket\") pod \"3599639b-5d4d-482f-b00f-24ec11e146e0\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") "
Sep 10 21:39:28.017280 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017016    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-certs\") pod \"3599639b-5d4d-482f-b00f-24ec11e146e0\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") "
Sep 10 21:39:28.017280 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017037    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-envoy\") pod \"3599639b-5d4d-482f-b00f-24ec11e146e0\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") "
Sep 10 21:39:28.017280 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017061    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2h578\" (UniqueName: \"kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-kube-api-access-2h578\") pod \"3599639b-5d4d-482f-b00f-24ec11e146e0\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") "
Sep 10 21:39:28.017280 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017076    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3599639b-5d4d-482f-b00f-24ec11e146e0-istiod-ca-cert\") pod \"3599639b-5d4d-482f-b00f-24ec11e146e0\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") "
Sep 10 21:39:28.017280 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017117    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-podinfo\") pod \"3599639b-5d4d-482f-b00f-24ec11e146e0\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") "
Sep 10 21:39:28.017522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017249    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3599639b-5d4d-482f-b00f-24ec11e146e0" (UID: "3599639b-5d4d-482f-b00f-24ec11e146e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:39:28.017522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017311    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3599639b-5d4d-482f-b00f-24ec11e146e0" (UID: "3599639b-5d4d-482f-b00f-24ec11e146e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:39:28.017522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017334    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-data\") pod \"3599639b-5d4d-482f-b00f-24ec11e146e0\" (UID: \"3599639b-5d4d-482f-b00f-24ec11e146e0\") "
Sep 10 21:39:28.017522 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017495    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3599639b-5d4d-482f-b00f-24ec11e146e0" (UID: "3599639b-5d4d-482f-b00f-24ec11e146e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:39:28.017719 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017632    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-data" (OuterVolumeSpecName: "istio-data") pod "3599639b-5d4d-482f-b00f-24ec11e146e0" (UID: "3599639b-5d4d-482f-b00f-24ec11e146e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:39:28.017719 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017649    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3599639b-5d4d-482f-b00f-24ec11e146e0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3599639b-5d4d-482f-b00f-24ec11e146e0" (UID: "3599639b-5d4d-482f-b00f-24ec11e146e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:39:28.017862 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017743    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-data\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:39:28.017862 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017760    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-credential-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:39:28.017862 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017769    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-socket\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:39:28.017862 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017805    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-workload-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:39:28.017862 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.017817    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3599639b-5d4d-482f-b00f-24ec11e146e0-istiod-ca-cert\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:39:28.019477 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.019453    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3599639b-5d4d-482f-b00f-24ec11e146e0" (UID: "3599639b-5d4d-482f-b00f-24ec11e146e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:39:28.019477 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.019468    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-kube-api-access-2h578" (OuterVolumeSpecName: "kube-api-access-2h578") pod "3599639b-5d4d-482f-b00f-24ec11e146e0" (UID: "3599639b-5d4d-482f-b00f-24ec11e146e0"). InnerVolumeSpecName "kube-api-access-2h578". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:39:28.019765 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.019738    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3599639b-5d4d-482f-b00f-24ec11e146e0" (UID: "3599639b-5d4d-482f-b00f-24ec11e146e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:39:28.019880 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.019743    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-token" (OuterVolumeSpecName: "istio-token") pod "3599639b-5d4d-482f-b00f-24ec11e146e0" (UID: "3599639b-5d4d-482f-b00f-24ec11e146e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:39:28.119109 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.119058    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-podinfo\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:39:28.119109 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.119099    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-token\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:39:28.119109 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.119108    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3599639b-5d4d-482f-b00f-24ec11e146e0-istio-envoy\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:39:28.119109 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.119116    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h578\" (UniqueName: \"kubernetes.io/projected/3599639b-5d4d-482f-b00f-24ec11e146e0-kube-api-access-2h578\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:39:28.622912 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.622876    2573 generic.go:356] "Generic (PLEG): container finished" podID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerID="f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1" exitCode=0
Sep 10 21:39:28.623105 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.622933    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"
Sep 10 21:39:28.623105 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.622959    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" event={"ID":"3599639b-5d4d-482f-b00f-24ec11e146e0","Type":"ContainerDied","Data":"f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1"}
Sep 10 21:39:28.623105 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.623018    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2" event={"ID":"3599639b-5d4d-482f-b00f-24ec11e146e0","Type":"ContainerDied","Data":"c4a0e8c4c144979ab7125e1c72772bc6cf450563fb9e465671bbdfa25b95559a"}
Sep 10 21:39:28.623105 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.623043    2573 scope.go:117] "RemoveContainer" containerID="f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1"
Sep 10 21:39:28.639756 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.639735    2573 scope.go:117] "RemoveContainer" containerID="41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96"
Sep 10 21:39:28.648081 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.648056    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"]
Sep 10 21:39:28.648247 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.648234    2573 scope.go:117] "RemoveContainer" containerID="f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1"
Sep 10 21:39:28.648488 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:39:28.648471    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1\": container with ID starting with f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1 not found: ID does not exist" containerID="f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1"
Sep 10 21:39:28.648527 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.648496    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1"} err="failed to get container status \"f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1\": rpc error: code = NotFound desc = could not find container \"f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1\": container with ID starting with f3af1555b1a441e696ab32348a395e0b4b61b63bca524bc837248da6b1b532e1 not found: ID does not exist"
Sep 10 21:39:28.648527 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.648516    2573 scope.go:117] "RemoveContainer" containerID="41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96"
Sep 10 21:39:28.648806 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:39:28.648770    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96\": container with ID starting with 41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96 not found: ID does not exist" containerID="41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96"
Sep 10 21:39:28.648857 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.648815    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96"} err="failed to get container status \"41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96\": rpc error: code = NotFound desc = could not find container \"41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96\": container with ID starting with 41ac280e8d968470180597a2a2f0b9537c9fecf2a819d2648dd8aa6e3d7f7a96 not found: ID does not exist"
Sep 10 21:39:28.651758 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.651738    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1f3879-openshift-default-84f6b89ddb-qg4b2"]
Sep 10 21:39:28.679034 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:39:28.679007    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" path="/var/lib/kubelet/pods/3599639b-5d4d-482f-b00f-24ec11e146e0/volumes"
Sep 10 21:40:24.657861 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:40:24.657830    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:40:24.663390 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:40:24.663366    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:40:24.970703 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:40:24.970621    2573 scope.go:117] "RemoveContainer" containerID="5ae6d9cc18f6dccf93725b12acd8829b1cc965a0533ee1a2768a1f04e8723a98"
Sep 10 21:40:48.208309 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:40:48.208268    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:41:11.908241 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:41:11.908202    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:41:19.615173 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:41:19.615129    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:41:24.996705 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:41:24.996661    2573 scope.go:117] "RemoveContainer" containerID="f3103f6eb6ce84e4c073f71b118bc6b134b0c147fb0a6e7433010ea8aa1e7547"
Sep 10 21:41:25.005523 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:41:25.005504    2573 scope.go:117] "RemoveContainer" containerID="c9f2ec6ab13c1ec356cec7fba58f72857e1657138fd5455b5ce6f69cac47c5f3"
Sep 10 21:41:27.808108 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:41:27.808075    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:41:42.903772 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:41:42.903735    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:43:12.205228 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:12.205188    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:43:18.502902 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:18.502826    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:43:26.209297 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:26.209259    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:43:35.907519 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.907484    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"]
Sep 10 21:43:35.907996 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.907974    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="521c1595-02fa-4f48-8054-aee7cc84ae01" containerName="revoke-keys"
Sep 10 21:43:35.907996 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.907998    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="521c1595-02fa-4f48-8054-aee7cc84ae01" containerName="revoke-keys"
Sep 10 21:43:35.907996 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.908016    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy"
Sep 10 21:43:35.908222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.908022    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy"
Sep 10 21:43:35.908222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.908041    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy"
Sep 10 21:43:35.908222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.908047    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy"
Sep 10 21:43:35.908222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.908117    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1418d7f5-81a7-406c-a204-f5afbe440779" containerName="istio-proxy"
Sep 10 21:43:35.908222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.908126    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy"
Sep 10 21:43:35.908222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.908132    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3599639b-5d4d-482f-b00f-24ec11e146e0" containerName="istio-proxy"
Sep 10 21:43:35.908222 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.908138    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="521c1595-02fa-4f48-8054-aee7cc84ae01" containerName="revoke-keys"
Sep 10 21:43:35.911056 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.911040    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:35.913078 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.913053    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-161df7d2-kserve-self-signed-certs\""
Sep 10 21:43:35.919847 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:35.919760    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"]
Sep 10 21:43:36.003364 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.003322    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdlmq\" (UniqueName: \"kubernetes.io/projected/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-kube-api-access-wdlmq\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.003542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.003364    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tmp-dir\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.003542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.003452    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-home\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.003542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.003489    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-model-cache\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.003542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.003506    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-dshm\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.003682 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.003546    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tls-certs\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.105014 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.104975    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdlmq\" (UniqueName: \"kubernetes.io/projected/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-kube-api-access-wdlmq\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.105014 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.105017    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tmp-dir\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.105221 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.105047    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-home\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.105221 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.105074    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-model-cache\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.105221 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.105106    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-dshm\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.105352 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.105261    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tls-certs\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.105450 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.105431    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tmp-dir\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.105516 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.105476    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-model-cache\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.105572 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.105517    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-home\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.107360 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.107325    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-dshm\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.107615 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.107597    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tls-certs\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.112518 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.112501    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdlmq\" (UniqueName: \"kubernetes.io/projected/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-kube-api-access-wdlmq\") pod \"e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") " pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.222723 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.222639    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:36.491971 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.491943    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"]
Sep 10 21:43:36.492488 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:43:36.492455    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f468eef_3bf5_4e5d_9532_15295d1e7ccc.slice/crio-261013a33dc19c372f65a93466c1294923db7ca0f552c689b3ad45f5bbe62737 WatchSource:0}: Error finding container 261013a33dc19c372f65a93466c1294923db7ca0f552c689b3ad45f5bbe62737: Status 404 returned error can't find the container with id 261013a33dc19c372f65a93466c1294923db7ca0f552c689b3ad45f5bbe62737
Sep 10 21:43:36.938583 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.938542    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj" event={"ID":"8f468eef-3bf5-4e5d-9532-15295d1e7ccc","Type":"ContainerStarted","Data":"293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4"}
Sep 10 21:43:36.938583 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.938587    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj" event={"ID":"8f468eef-3bf5-4e5d-9532-15295d1e7ccc","Type":"ContainerStarted","Data":"261013a33dc19c372f65a93466c1294923db7ca0f552c689b3ad45f5bbe62737"}
Sep 10 21:43:36.956151 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:36.956102    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj" podStartSLOduration=1.956087914 podStartE2EDuration="1.956087914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:43:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:43:36.953995105 +0000 UTC m=+1992.887661920" watchObservedRunningTime="2026-09-10 21:43:36.956087914 +0000 UTC m=+1992.889754728"
Sep 10 21:43:38.507594 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:38.507556    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:43:46.119295 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.119258    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"]
Sep 10 21:43:46.122676 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.122648    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.124992 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.124962    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-161df7d2-kserve-self-signed-certs\""
Sep 10 21:43:46.131229 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.131200    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"]
Sep 10 21:43:46.214741 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.214695    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-model-cache\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.214741 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.214739    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/972066fa-99a2-4b2c-921d-e12239e1f192-tls-certs\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.215051 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.214805    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-tmp-dir\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.215051 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.214864    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffl9b\" (UniqueName: \"kubernetes.io/projected/972066fa-99a2-4b2c-921d-e12239e1f192-kube-api-access-ffl9b\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.215051 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.214885    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-dshm\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.215051 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.214914    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-home\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.223237 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.223211    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:46.223358 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.223255    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:46.235761 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.235734    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:46.315689 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.315655    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/972066fa-99a2-4b2c-921d-e12239e1f192-tls-certs\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.315914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.315725    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-tmp-dir\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.315914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.315844    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffl9b\" (UniqueName: \"kubernetes.io/projected/972066fa-99a2-4b2c-921d-e12239e1f192-kube-api-access-ffl9b\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.315914 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.315875    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-dshm\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.316098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.315921    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-home\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.316098 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.315953    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-model-cache\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.316214 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.316135    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-tmp-dir\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.316373 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.316350    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-model-cache\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.316417 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.316375    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-home\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.318160 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.318142    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-dshm\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.318546 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.318525    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/972066fa-99a2-4b2c-921d-e12239e1f192-tls-certs\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.323312 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.323286    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffl9b\" (UniqueName: \"kubernetes.io/projected/972066fa-99a2-4b2c-921d-e12239e1f192-kube-api-access-ffl9b\") pod \"e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") " pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.432748 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.432656    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:46.704968 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.704941    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"]
Sep 10 21:43:46.707637 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:43:46.707609    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod972066fa_99a2_4b2c_921d_e12239e1f192.slice/crio-cd519826f9afbe6246c49b8c141705df32bd1b99f4d6053f68725d6283a27475 WatchSource:0}: Error finding container cd519826f9afbe6246c49b8c141705df32bd1b99f4d6053f68725d6283a27475: Status 404 returned error can't find the container with id cd519826f9afbe6246c49b8c141705df32bd1b99f4d6053f68725d6283a27475
Sep 10 21:43:46.993577 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.993485    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289" event={"ID":"972066fa-99a2-4b2c-921d-e12239e1f192","Type":"ContainerStarted","Data":"ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43"}
Sep 10 21:43:46.993577 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:46.993531    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289" event={"ID":"972066fa-99a2-4b2c-921d-e12239e1f192","Type":"ContainerStarted","Data":"cd519826f9afbe6246c49b8c141705df32bd1b99f4d6053f68725d6283a27475"}
Sep 10 21:43:47.005120 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:47.005096    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:43:47.011555 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:47.011511    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289" podStartSLOduration=1.011497578 podStartE2EDuration="1.011497578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:43:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:43:47.010063793 +0000 UTC m=+2002.943730611" watchObservedRunningTime="2026-09-10 21:43:47.011497578 +0000 UTC m=+2002.945164393"
Sep 10 21:43:49.013979 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:49.013944    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:43:56.433074 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:56.433037    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:56.433560 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:56.433297    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:56.445664 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:56.445641    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:43:57.057955 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:43:57.057928    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:44:07.538998 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:07.538966    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"]
Sep 10 21:44:08.294167 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:08.294129    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"]
Sep 10 21:44:10.718032 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:10.717996    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:44:20.116150 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:20.116107    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:44:22.558766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.558730    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj" podUID="8f468eef-3bf5-4e5d-9532-15295d1e7ccc" containerName="main" containerID="cri-o://293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4" gracePeriod=45
Sep 10 21:44:22.800226 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.800203    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:44:22.945738 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.945704    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tls-certs\") pod \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") "
Sep 10 21:44:22.945926 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.945793    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tmp-dir\") pod \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") "
Sep 10 21:44:22.945926 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.945827    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-model-cache\") pod \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") "
Sep 10 21:44:22.945926 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.945874    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-home\") pod \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") "
Sep 10 21:44:22.945926 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.945910    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdlmq\" (UniqueName: \"kubernetes.io/projected/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-kube-api-access-wdlmq\") pod \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") "
Sep 10 21:44:22.946154 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.945957    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-dshm\") pod \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\" (UID: \"8f468eef-3bf5-4e5d-9532-15295d1e7ccc\") "
Sep 10 21:44:22.946154 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.946083    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-model-cache" (OuterVolumeSpecName: "model-cache") pod "8f468eef-3bf5-4e5d-9532-15295d1e7ccc" (UID: "8f468eef-3bf5-4e5d-9532-15295d1e7ccc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:22.946253 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.946153    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8f468eef-3bf5-4e5d-9532-15295d1e7ccc" (UID: "8f468eef-3bf5-4e5d-9532-15295d1e7ccc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:22.946253 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.946171    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-home" (OuterVolumeSpecName: "home") pod "8f468eef-3bf5-4e5d-9532-15295d1e7ccc" (UID: "8f468eef-3bf5-4e5d-9532-15295d1e7ccc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:22.946253 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.946247    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:22.946409 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.946266    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:22.947994 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.947973    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8f468eef-3bf5-4e5d-9532-15295d1e7ccc" (UID: "8f468eef-3bf5-4e5d-9532-15295d1e7ccc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:44:22.948094 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.948013    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-kube-api-access-wdlmq" (OuterVolumeSpecName: "kube-api-access-wdlmq") pod "8f468eef-3bf5-4e5d-9532-15295d1e7ccc" (UID: "8f468eef-3bf5-4e5d-9532-15295d1e7ccc"). InnerVolumeSpecName "kube-api-access-wdlmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:44:22.948094 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:22.948069    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-dshm" (OuterVolumeSpecName: "dshm") pod "8f468eef-3bf5-4e5d-9532-15295d1e7ccc" (UID: "8f468eef-3bf5-4e5d-9532-15295d1e7ccc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:23.047760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.047699    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:23.047760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.047756    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdlmq\" (UniqueName: \"kubernetes.io/projected/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-kube-api-access-wdlmq\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:23.047760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.047767    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:23.047760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.047794    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f468eef-3bf5-4e5d-9532-15295d1e7ccc-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:23.183486 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.183451    2573 generic.go:356] "Generic (PLEG): container finished" podID="8f468eef-3bf5-4e5d-9532-15295d1e7ccc" containerID="293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4" exitCode=0
Sep 10 21:44:23.183659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.183495    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"
Sep 10 21:44:23.183659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.183538    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj" event={"ID":"8f468eef-3bf5-4e5d-9532-15295d1e7ccc","Type":"ContainerDied","Data":"293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4"}
Sep 10 21:44:23.183659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.183580    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj" event={"ID":"8f468eef-3bf5-4e5d-9532-15295d1e7ccc","Type":"ContainerDied","Data":"261013a33dc19c372f65a93466c1294923db7ca0f552c689b3ad45f5bbe62737"}
Sep 10 21:44:23.183659 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.183598    2573 scope.go:117] "RemoveContainer" containerID="293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4"
Sep 10 21:44:23.192703 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.192689    2573 scope.go:117] "RemoveContainer" containerID="293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4"
Sep 10 21:44:23.193049 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:44:23.193031    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4\": container with ID starting with 293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4 not found: ID does not exist" containerID="293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4"
Sep 10 21:44:23.193108 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.193057    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4"} err="failed to get container status \"293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4\": rpc error: code = NotFound desc = could not find container \"293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4\": container with ID starting with 293e560a012864ea7cb23e193828e0fa2b5a83a90b346ff1f84b37a9f2a267f4 not found: ID does not exist"
Sep 10 21:44:23.206990 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.206930    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"]
Sep 10 21:44:23.209046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.209023    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-161df7d2-kserve-7b646d9dc9-x2rzj"]
Sep 10 21:44:23.313253 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.313207    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289" podUID="972066fa-99a2-4b2c-921d-e12239e1f192" containerName="main" containerID="cri-o://ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43" gracePeriod=45
Sep 10 21:44:23.547362 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.547336    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:44:23.653433 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.653389    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-home\") pod \"972066fa-99a2-4b2c-921d-e12239e1f192\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") "
Sep 10 21:44:23.653929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.653451    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-dshm\") pod \"972066fa-99a2-4b2c-921d-e12239e1f192\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") "
Sep 10 21:44:23.653929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.653473    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/972066fa-99a2-4b2c-921d-e12239e1f192-tls-certs\") pod \"972066fa-99a2-4b2c-921d-e12239e1f192\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") "
Sep 10 21:44:23.653929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.653506    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-tmp-dir\") pod \"972066fa-99a2-4b2c-921d-e12239e1f192\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") "
Sep 10 21:44:23.653929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.653526    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-model-cache\") pod \"972066fa-99a2-4b2c-921d-e12239e1f192\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") "
Sep 10 21:44:23.653929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.653551    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffl9b\" (UniqueName: \"kubernetes.io/projected/972066fa-99a2-4b2c-921d-e12239e1f192-kube-api-access-ffl9b\") pod \"972066fa-99a2-4b2c-921d-e12239e1f192\" (UID: \"972066fa-99a2-4b2c-921d-e12239e1f192\") "
Sep 10 21:44:23.653929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.653742    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-home" (OuterVolumeSpecName: "home") pod "972066fa-99a2-4b2c-921d-e12239e1f192" (UID: "972066fa-99a2-4b2c-921d-e12239e1f192"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:23.653929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.653757    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "972066fa-99a2-4b2c-921d-e12239e1f192" (UID: "972066fa-99a2-4b2c-921d-e12239e1f192"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:23.653929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.653836    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-model-cache" (OuterVolumeSpecName: "model-cache") pod "972066fa-99a2-4b2c-921d-e12239e1f192" (UID: "972066fa-99a2-4b2c-921d-e12239e1f192"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:23.655771 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.655739    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/972066fa-99a2-4b2c-921d-e12239e1f192-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "972066fa-99a2-4b2c-921d-e12239e1f192" (UID: "972066fa-99a2-4b2c-921d-e12239e1f192"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:44:23.655771 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.655737    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-dshm" (OuterVolumeSpecName: "dshm") pod "972066fa-99a2-4b2c-921d-e12239e1f192" (UID: "972066fa-99a2-4b2c-921d-e12239e1f192"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:23.655959 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.655853    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/972066fa-99a2-4b2c-921d-e12239e1f192-kube-api-access-ffl9b" (OuterVolumeSpecName: "kube-api-access-ffl9b") pod "972066fa-99a2-4b2c-921d-e12239e1f192" (UID: "972066fa-99a2-4b2c-921d-e12239e1f192"). InnerVolumeSpecName "kube-api-access-ffl9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:44:23.755027 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.754939    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-home\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:23.755027 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.754971    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-dshm\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:23.755027 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.754981    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/972066fa-99a2-4b2c-921d-e12239e1f192-tls-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:23.755027 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.754991    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-tmp-dir\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:23.755027 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.755000    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/972066fa-99a2-4b2c-921d-e12239e1f192-model-cache\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:23.755027 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:23.755009    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffl9b\" (UniqueName: \"kubernetes.io/projected/972066fa-99a2-4b2c-921d-e12239e1f192-kube-api-access-ffl9b\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:24.191864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.191829    2573 generic.go:356] "Generic (PLEG): container finished" podID="972066fa-99a2-4b2c-921d-e12239e1f192" containerID="ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43" exitCode=0
Sep 10 21:44:24.192045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.191878    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"
Sep 10 21:44:24.192045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.191927    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289" event={"ID":"972066fa-99a2-4b2c-921d-e12239e1f192","Type":"ContainerDied","Data":"ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43"}
Sep 10 21:44:24.192045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.191975    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289" event={"ID":"972066fa-99a2-4b2c-921d-e12239e1f192","Type":"ContainerDied","Data":"cd519826f9afbe6246c49b8c141705df32bd1b99f4d6053f68725d6283a27475"}
Sep 10 21:44:24.192045 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.191993    2573 scope.go:117] "RemoveContainer" containerID="ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43"
Sep 10 21:44:24.200993 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.200979    2573 scope.go:117] "RemoveContainer" containerID="ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43"
Sep 10 21:44:24.201273 ip-10-0-135-74 kubenswrapper[2573]: E0910 21:44:24.201254    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43\": container with ID starting with ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43 not found: ID does not exist" containerID="ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43"
Sep 10 21:44:24.201324 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.201281    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43"} err="failed to get container status \"ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43\": rpc error: code = NotFound desc = could not find container \"ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43\": container with ID starting with ab71a0253278a439b4321263ce0dbfb2b5084a629969d135d8d5857c7fe04b43 not found: ID does not exist"
Sep 10 21:44:24.214978 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.214931    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"]
Sep 10 21:44:24.216965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.216947    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-161df7d2-kserve-7cbcd5c7f5-nb289"]
Sep 10 21:44:24.678832 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.678799    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f468eef-3bf5-4e5d-9532-15295d1e7ccc" path="/var/lib/kubelet/pods/8f468eef-3bf5-4e5d-9532-15295d1e7ccc/volumes"
Sep 10 21:44:24.679209 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:24.679195    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="972066fa-99a2-4b2c-921d-e12239e1f192" path="/var/lib/kubelet/pods/972066fa-99a2-4b2c-921d-e12239e1f192/volumes"
Sep 10 21:44:28.117067 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.117029    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:44:28.292899 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.292862    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"]
Sep 10 21:44:28.293297 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.293277    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="972066fa-99a2-4b2c-921d-e12239e1f192" containerName="main"
Sep 10 21:44:28.293392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.293298    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="972066fa-99a2-4b2c-921d-e12239e1f192" containerName="main"
Sep 10 21:44:28.293392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.293328    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f468eef-3bf5-4e5d-9532-15295d1e7ccc" containerName="main"
Sep 10 21:44:28.293392 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.293336    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f468eef-3bf5-4e5d-9532-15295d1e7ccc" containerName="main"
Sep 10 21:44:28.293550 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.293439    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="972066fa-99a2-4b2c-921d-e12239e1f192" containerName="main"
Sep 10 21:44:28.293550 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.293453    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f468eef-3bf5-4e5d-9532-15295d1e7ccc" containerName="main"
Sep 10 21:44:28.296552 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.296530    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.298578 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.298558    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:44:28.299223 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.299200    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 21:44:28.299351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.299248    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-99rzw\""
Sep 10 21:44:28.299351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.299275    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:44:28.299351 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.299202    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:44:28.305853 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.305831    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"]
Sep 10 21:44:28.391819 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.391707    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl2zw\" (UniqueName: \"kubernetes.io/projected/459f8d4e-6795-4672-9660-1078fb379a84-kube-api-access-hl2zw\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.391819 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.391768    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-metrics-certs\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.392037 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.391861    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-maas-api-tls\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.392037 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.391880    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/459f8d4e-6795-4672-9660-1078fb379a84-openshift-service-ca\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.493138 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.493101    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-metrics-certs\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.493323 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.493169    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-maas-api-tls\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.493323 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.493196    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/459f8d4e-6795-4672-9660-1078fb379a84-openshift-service-ca\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.493323 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.493244    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl2zw\" (UniqueName: \"kubernetes.io/projected/459f8d4e-6795-4672-9660-1078fb379a84-kube-api-access-hl2zw\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.493916 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.493884    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/459f8d4e-6795-4672-9660-1078fb379a84-openshift-service-ca\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.495602 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.495575    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-metrics-certs\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.495818 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.495797    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-maas-api-tls\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.500675 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.500649    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl2zw\" (UniqueName: \"kubernetes.io/projected/459f8d4e-6795-4672-9660-1078fb379a84-kube-api-access-hl2zw\") pod \"maas-api-5b8f865cbb-8vzw5\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.608322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.608281    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:28.887478 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.887452    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"]
Sep 10 21:44:28.888689 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:44:28.888661    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod459f8d4e_6795_4672_9660_1078fb379a84.slice/crio-352909fa77456da939b1a8bef4098719cd48ec5e420ae404a192e1f818a0cbf8 WatchSource:0}: Error finding container 352909fa77456da939b1a8bef4098719cd48ec5e420ae404a192e1f818a0cbf8: Status 404 returned error can't find the container with id 352909fa77456da939b1a8bef4098719cd48ec5e420ae404a192e1f818a0cbf8
Sep 10 21:44:28.890053 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:28.890035    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:44:29.219171 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:29.219137    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5" event={"ID":"459f8d4e-6795-4672-9660-1078fb379a84","Type":"ContainerStarted","Data":"352909fa77456da939b1a8bef4098719cd48ec5e420ae404a192e1f818a0cbf8"}
Sep 10 21:44:30.225729 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:30.225686    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5" event={"ID":"459f8d4e-6795-4672-9660-1078fb379a84","Type":"ContainerStarted","Data":"b120098bc0d160d3e2216d06af6666bd737f7b36536c2351a1e758babb6645e4"}
Sep 10 21:44:30.226178 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:30.225757    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:30.243471 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:30.243423    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5" podStartSLOduration=1.790211985 podStartE2EDuration="2.243409967s" totalImagesPullingTime="453.197982ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:44:30.241873768 +0000 UTC m=+2046.175540582" watchObservedRunningTime="2026-09-10 21:44:30.243409967 +0000 UTC m=+2046.177076799"
Sep 10 21:44:35.612088 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:35.612046    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:44:36.236407 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:36.236375    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:42.414878 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:42.414833    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:44:44.120414 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.120369    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"]
Sep 10 21:44:44.120910 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.120680    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5" podUID="459f8d4e-6795-4672-9660-1078fb379a84" containerName="maas-api" containerID="cri-o://b120098bc0d160d3e2216d06af6666bd737f7b36536c2351a1e758babb6645e4" gracePeriod=30
Sep 10 21:44:44.303303 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.303259    2573 generic.go:356] "Generic (PLEG): container finished" podID="459f8d4e-6795-4672-9660-1078fb379a84" containerID="b120098bc0d160d3e2216d06af6666bd737f7b36536c2351a1e758babb6645e4" exitCode=0
Sep 10 21:44:44.303434 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.303326    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5" event={"ID":"459f8d4e-6795-4672-9660-1078fb379a84","Type":"ContainerDied","Data":"b120098bc0d160d3e2216d06af6666bd737f7b36536c2351a1e758babb6645e4"}
Sep 10 21:44:44.363487 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.363458    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:44.539686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.539596    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl2zw\" (UniqueName: \"kubernetes.io/projected/459f8d4e-6795-4672-9660-1078fb379a84-kube-api-access-hl2zw\") pod \"459f8d4e-6795-4672-9660-1078fb379a84\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") "
Sep 10 21:44:44.539686 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.539650    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/459f8d4e-6795-4672-9660-1078fb379a84-openshift-service-ca\") pod \"459f8d4e-6795-4672-9660-1078fb379a84\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") "
Sep 10 21:44:44.539978 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.539696    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-maas-api-tls\") pod \"459f8d4e-6795-4672-9660-1078fb379a84\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") "
Sep 10 21:44:44.539978 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.539728    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-metrics-certs\") pod \"459f8d4e-6795-4672-9660-1078fb379a84\" (UID: \"459f8d4e-6795-4672-9660-1078fb379a84\") "
Sep 10 21:44:44.540169 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.540144    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/459f8d4e-6795-4672-9660-1078fb379a84-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "459f8d4e-6795-4672-9660-1078fb379a84" (UID: "459f8d4e-6795-4672-9660-1078fb379a84"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:44:44.542078 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.542045    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "459f8d4e-6795-4672-9660-1078fb379a84" (UID: "459f8d4e-6795-4672-9660-1078fb379a84"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:44:44.542184 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.542067    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "459f8d4e-6795-4672-9660-1078fb379a84" (UID: "459f8d4e-6795-4672-9660-1078fb379a84"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:44:44.542184 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.542076    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/459f8d4e-6795-4672-9660-1078fb379a84-kube-api-access-hl2zw" (OuterVolumeSpecName: "kube-api-access-hl2zw") pod "459f8d4e-6795-4672-9660-1078fb379a84" (UID: "459f8d4e-6795-4672-9660-1078fb379a84"). InnerVolumeSpecName "kube-api-access-hl2zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:44:44.641532 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.641494    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-maas-api-tls\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:44.641532 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.641523    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/459f8d4e-6795-4672-9660-1078fb379a84-metrics-certs\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:44.641532 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.641533    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl2zw\" (UniqueName: \"kubernetes.io/projected/459f8d4e-6795-4672-9660-1078fb379a84-kube-api-access-hl2zw\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:44.641532 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:44.641542    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/459f8d4e-6795-4672-9660-1078fb379a84-openshift-service-ca\") on node \"ip-10-0-135-74.ec2.internal\" DevicePath \"\""
Sep 10 21:44:45.309514 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:45.309483    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5" event={"ID":"459f8d4e-6795-4672-9660-1078fb379a84","Type":"ContainerDied","Data":"352909fa77456da939b1a8bef4098719cd48ec5e420ae404a192e1f818a0cbf8"}
Sep 10 21:44:45.309947 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:45.309518    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"
Sep 10 21:44:45.309947 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:45.309543    2573 scope.go:117] "RemoveContainer" containerID="b120098bc0d160d3e2216d06af6666bd737f7b36536c2351a1e758babb6645e4"
Sep 10 21:44:45.331685 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:45.331654    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"]
Sep 10 21:44:45.335848 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:45.335824    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-8vzw5"]
Sep 10 21:44:46.678687 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:46.678605    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="459f8d4e-6795-4672-9660-1078fb379a84" path="/var/lib/kubelet/pods/459f8d4e-6795-4672-9660-1078fb379a84/volumes"
Sep 10 21:44:48.406201 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:48.406167    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:44:55.018590 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:55.018550    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4nr74"]
Sep 10 21:44:57.904725 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:44:57.904692    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6454784d76-w2h5d_b82d9d86-3adf-4478-beee-5f2145c09030/authorino/0.log"
Sep 10 21:45:00.133125 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:00.133089    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817945-ccrx4"]
Sep 10 21:45:00.133506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:00.133422    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="459f8d4e-6795-4672-9660-1078fb379a84" containerName="maas-api"
Sep 10 21:45:00.133506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:00.133433    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="459f8d4e-6795-4672-9660-1078fb379a84" containerName="maas-api"
Sep 10 21:45:00.133506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:00.133507    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="459f8d4e-6795-4672-9660-1078fb379a84" containerName="maas-api"
Sep 10 21:45:00.136452 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:00.136436    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-ccrx4"
Sep 10 21:45:00.138511 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:00.138483    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:45:00.142455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:00.142428    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817945-ccrx4"]
Sep 10 21:45:00.147515 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:00.147498    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-ccrx4"
Sep 10 21:45:00.417937 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:00.417910    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817945-ccrx4"]
Sep 10 21:45:00.419917 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:45:00.419885    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2b977a_abd0_41c4_b5a4_a268aecff99e.slice/crio-4118b0b14c29b21df98584301fe726f4395bf724a2d9b381f876188e0081d9af WatchSource:0}: Error finding container 4118b0b14c29b21df98584301fe726f4395bf724a2d9b381f876188e0081d9af: Status 404 returned error can't find the container with id 4118b0b14c29b21df98584301fe726f4395bf724a2d9b381f876188e0081d9af
Sep 10 21:45:01.402506 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:01.402467    2573 generic.go:356] "Generic (PLEG): container finished" podID="bb2b977a-abd0-41c4-b5a4-a268aecff99e" containerID="e711e27f6bf8733d3102bbdb4f0132483df87ebe7c87da70b01642a000a29866" exitCode=0
Sep 10 21:45:01.402937 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:01.402570    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-ccrx4" event={"ID":"bb2b977a-abd0-41c4-b5a4-a268aecff99e","Type":"ContainerDied","Data":"e711e27f6bf8733d3102bbdb4f0132483df87ebe7c87da70b01642a000a29866"}
Sep 10 21:45:01.402937 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:01.402651    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-ccrx4" event={"ID":"bb2b977a-abd0-41c4-b5a4-a268aecff99e","Type":"ContainerStarted","Data":"4118b0b14c29b21df98584301fe726f4395bf724a2d9b381f876188e0081d9af"}
Sep 10 21:45:02.534767 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:02.534739    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-ccrx4"
Sep 10 21:45:03.414249 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:03.414218    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-ccrx4"
Sep 10 21:45:03.414428 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:03.414263    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-ccrx4" event={"ID":"bb2b977a-abd0-41c4-b5a4-a268aecff99e","Type":"ContainerDied","Data":"4118b0b14c29b21df98584301fe726f4395bf724a2d9b381f876188e0081d9af"}
Sep 10 21:45:03.414428 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:03.414320    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4118b0b14c29b21df98584301fe726f4395bf724a2d9b381f876188e0081d9af"
Sep 10 21:45:03.872864 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:03.872837    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd_8db3bd51-745a-4ec9-9c39-4145c3a43dac/copy-manifests/0.log"
Sep 10 21:45:03.880909 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:03.880885    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd_8db3bd51-745a-4ec9-9c39-4145c3a43dac/manager/0.log"
Sep 10 21:45:04.418919 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:04.418886    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817930-8x79g_6808a55b-8d58-43fa-a2ea-70428653e272/cleanup/0.log"
Sep 10 21:45:04.528719 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:04.528690    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817945-ccrx4_bb2b977a-abd0-41c4-b5a4-a268aecff99e/cleanup/0.log"
Sep 10 21:45:05.365760 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:05.365727    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6454784d76-w2h5d_b82d9d86-3adf-4478-beee-5f2145c09030/authorino/0.log"
Sep 10 21:45:05.685158 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:05.685082    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-tdbzm_8b887163-5548-4810-b818-aaf178145ed4/kuadrant-console-plugin/0.log"
Sep 10 21:45:05.926041 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:05.926013    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-4nr74_220ecdaf-58d6-4f99-bedc-eb3ed2906d42/limitador/0.log"
Sep 10 21:45:06.715511 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:06.715482    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545c498cb-gc9s9_79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d/kube-auth-proxy/0.log"
Sep 10 21:45:07.840597 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:07.840563    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hgpnd_caee352d-a74c-4581-a25c-eb44823eb5c6/main/0.log"
Sep 10 21:45:13.317479 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.317443    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vms7j/must-gather-fh7rf"]
Sep 10 21:45:13.317873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.317817    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb2b977a-abd0-41c4-b5a4-a268aecff99e" containerName="cleanup"
Sep 10 21:45:13.317873 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.317831    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb2b977a-abd0-41c4-b5a4-a268aecff99e" containerName="cleanup"
Sep 10 21:45:13.317951 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.317898    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb2b977a-abd0-41c4-b5a4-a268aecff99e" containerName="cleanup"
Sep 10 21:45:13.320965 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.320944    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vms7j/must-gather-fh7rf"
Sep 10 21:45:13.323277 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.323256    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vms7j\"/\"default-dockercfg-jtgj9\""
Sep 10 21:45:13.323397 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.323274    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vms7j\"/\"openshift-service-ca.crt\""
Sep 10 21:45:13.323978 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.323962    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vms7j\"/\"kube-root-ca.crt\""
Sep 10 21:45:13.330631 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.330613    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vms7j/must-gather-fh7rf"]
Sep 10 21:45:13.388942 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.388907    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmv9p\" (UniqueName: \"kubernetes.io/projected/ada8a508-b23f-40b8-9339-b79747ff5d84-kube-api-access-wmv9p\") pod \"must-gather-fh7rf\" (UID: \"ada8a508-b23f-40b8-9339-b79747ff5d84\") " pod="openshift-must-gather-vms7j/must-gather-fh7rf"
Sep 10 21:45:13.389138 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.388977    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ada8a508-b23f-40b8-9339-b79747ff5d84-must-gather-output\") pod \"must-gather-fh7rf\" (UID: \"ada8a508-b23f-40b8-9339-b79747ff5d84\") " pod="openshift-must-gather-vms7j/must-gather-fh7rf"
Sep 10 21:45:13.489572 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.489541    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmv9p\" (UniqueName: \"kubernetes.io/projected/ada8a508-b23f-40b8-9339-b79747ff5d84-kube-api-access-wmv9p\") pod \"must-gather-fh7rf\" (UID: \"ada8a508-b23f-40b8-9339-b79747ff5d84\") " pod="openshift-must-gather-vms7j/must-gather-fh7rf"
Sep 10 21:45:13.489876 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.489592    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ada8a508-b23f-40b8-9339-b79747ff5d84-must-gather-output\") pod \"must-gather-fh7rf\" (UID: \"ada8a508-b23f-40b8-9339-b79747ff5d84\") " pod="openshift-must-gather-vms7j/must-gather-fh7rf"
Sep 10 21:45:13.489939 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.489916    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ada8a508-b23f-40b8-9339-b79747ff5d84-must-gather-output\") pod \"must-gather-fh7rf\" (UID: \"ada8a508-b23f-40b8-9339-b79747ff5d84\") " pod="openshift-must-gather-vms7j/must-gather-fh7rf"
Sep 10 21:45:13.498572 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.498549    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmv9p\" (UniqueName: \"kubernetes.io/projected/ada8a508-b23f-40b8-9339-b79747ff5d84-kube-api-access-wmv9p\") pod \"must-gather-fh7rf\" (UID: \"ada8a508-b23f-40b8-9339-b79747ff5d84\") " pod="openshift-must-gather-vms7j/must-gather-fh7rf"
Sep 10 21:45:13.631355 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.631321    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vms7j/must-gather-fh7rf"
Sep 10 21:45:13.904929 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:13.904901    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vms7j/must-gather-fh7rf"]
Sep 10 21:45:13.907233 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:45:13.907205    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podada8a508_b23f_40b8_9339_b79747ff5d84.slice/crio-e9ec2df1a770712e149a201dd98001e803d0dc930aee4aaaf0199eb6269814f6 WatchSource:0}: Error finding container e9ec2df1a770712e149a201dd98001e803d0dc930aee4aaaf0199eb6269814f6: Status 404 returned error can't find the container with id e9ec2df1a770712e149a201dd98001e803d0dc930aee4aaaf0199eb6269814f6
Sep 10 21:45:14.481672 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:14.481619    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vms7j/must-gather-fh7rf" event={"ID":"ada8a508-b23f-40b8-9339-b79747ff5d84","Type":"ContainerStarted","Data":"e9ec2df1a770712e149a201dd98001e803d0dc930aee4aaaf0199eb6269814f6"}
Sep 10 21:45:15.492138 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:15.492100    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vms7j/must-gather-fh7rf" event={"ID":"ada8a508-b23f-40b8-9339-b79747ff5d84","Type":"ContainerStarted","Data":"452ff890396c199d687bce0fe08c5fb52c011a37e4eebba6a2c8c072aa8c4d86"}
Sep 10 21:45:15.492138 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:15.492148    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vms7j/must-gather-fh7rf" event={"ID":"ada8a508-b23f-40b8-9339-b79747ff5d84","Type":"ContainerStarted","Data":"b738f5f1801c671cd0064c683960bcd6443a1a3a53bf4520b9958fed0a3ce664"}
Sep 10 21:45:15.508458 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:15.508391    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vms7j/must-gather-fh7rf" podStartSLOduration=1.718286204 podStartE2EDuration="2.508370117s" totalImagesPullingTime="790.083913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:45:15.505708573 +0000 UTC m=+2091.439375388" watchObservedRunningTime="2026-09-10 21:45:15.508370117 +0000 UTC m=+2091.442036933"
Sep 10 21:45:16.384617 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:16.384581    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z8k59_1f372f51-ce6b-46c2-9438-df2f95851903/global-pull-secret-syncer/0.log"
Sep 10 21:45:16.520317 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:16.520282    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vttxv_3160b92d-6922-4d94-a5ed-b92338837c2f/konnectivity-agent/0.log"
Sep 10 21:45:16.650612 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:16.650532    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-74.ec2.internal_bcd962acf554a8e9018c7c906807b563/haproxy/0.log"
Sep 10 21:45:21.424094 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:21.424034    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6454784d76-w2h5d_b82d9d86-3adf-4478-beee-5f2145c09030/authorino/0.log"
Sep 10 21:45:21.523182 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:21.523147    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-tdbzm_8b887163-5548-4810-b818-aaf178145ed4/kuadrant-console-plugin/0.log"
Sep 10 21:45:21.594839 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:21.594804    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-4nr74_220ecdaf-58d6-4f99-bedc-eb3ed2906d42/limitador/0.log"
Sep 10 21:45:23.080830 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.080796    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0939e986-4979-4c34-a308-18c22729f75d/alertmanager/0.log"
Sep 10 21:45:23.104639 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.104612    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0939e986-4979-4c34-a308-18c22729f75d/config-reloader/0.log"
Sep 10 21:45:23.130735 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.130694    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0939e986-4979-4c34-a308-18c22729f75d/kube-rbac-proxy-web/0.log"
Sep 10 21:45:23.153791 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.153734    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0939e986-4979-4c34-a308-18c22729f75d/kube-rbac-proxy/0.log"
Sep 10 21:45:23.182149 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.182116    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0939e986-4979-4c34-a308-18c22729f75d/kube-rbac-proxy-metric/0.log"
Sep 10 21:45:23.209046 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.209021    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0939e986-4979-4c34-a308-18c22729f75d/prom-label-proxy/0.log"
Sep 10 21:45:23.239475 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.239451    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0939e986-4979-4c34-a308-18c22729f75d/init-config-reloader/0.log"
Sep 10 21:45:23.283922 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.283854    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-4nfrp_ce51c5d4-3d30-415d-8b3c-e009a99848ec/cluster-monitoring-operator/0.log"
Sep 10 21:45:23.567110 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.566182    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rfvqt_25b0113e-d86c-464d-89d7-c79c0a4754a9/node-exporter/0.log"
Sep 10 21:45:23.618414 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.618344    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rfvqt_25b0113e-d86c-464d-89d7-c79c0a4754a9/kube-rbac-proxy/0.log"
Sep 10 21:45:23.658725 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.658689    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rfvqt_25b0113e-d86c-464d-89d7-c79c0a4754a9/init-textfile/0.log"
Sep 10 21:45:23.784455 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.784357    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-ljcqx_8e637906-06fe-417c-8278-44962a5a80af/kube-rbac-proxy-main/0.log"
Sep 10 21:45:23.810743 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.810705    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-ljcqx_8e637906-06fe-417c-8278-44962a5a80af/kube-rbac-proxy-self/0.log"
Sep 10 21:45:23.838279 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:23.838242    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-ljcqx_8e637906-06fe-417c-8278-44962a5a80af/openshift-state-metrics/0.log"
Sep 10 21:45:24.064809 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:24.064705    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-25sjm_2d301662-abdf-41e0-901a-4a171c47e84a/prometheus-operator/0.log"
Sep 10 21:45:24.089716 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:24.089676    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-25sjm_2d301662-abdf-41e0-901a-4a171c47e84a/kube-rbac-proxy/0.log"
Sep 10 21:45:24.155420 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:24.155393    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-774d68476c-v4dqc_a5f04b3f-c716-4d0f-89d0-7a81d7cab965/telemeter-client/0.log"
Sep 10 21:45:24.183129 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:24.183083    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-774d68476c-v4dqc_a5f04b3f-c716-4d0f-89d0-7a81d7cab965/reload/0.log"
Sep 10 21:45:24.206268 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:24.206224    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-774d68476c-v4dqc_a5f04b3f-c716-4d0f-89d0-7a81d7cab965/kube-rbac-proxy/0.log"
Sep 10 21:45:24.715206 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:24.714635    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:45:24.720149 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:24.720048    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:45:25.064473 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:25.064379    2573 scope.go:117] "RemoveContainer" containerID="521e85dc16ffd0fa3c12f8855cc09601188a2ac7c7caa013d3443529246dd745"
Sep 10 21:45:25.075542 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:25.075518    2573 scope.go:117] "RemoveContainer" containerID="2ad76c40d55a0ee581af637c8c7505bb2a804e523ea425ff3a6d31fc220f94cd"
Sep 10 21:45:26.416907 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.416874    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/1.log"
Sep 10 21:45:26.422637 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.422600    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-4d785_bef584d6-f102-4e09-998f-2622be652abc/console-operator/2.log"
Sep 10 21:45:26.504904 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.504860    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"]
Sep 10 21:45:26.511885 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.511853    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.519125 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.519097    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"]
Sep 10 21:45:26.634567 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.634523    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-sys\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.634766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.634575    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-proc\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.634766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.634600    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-podres\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.634766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.634654    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw97h\" (UniqueName: \"kubernetes.io/projected/4b8a71e6-a72c-4731-b732-5600e9e288b7-kube-api-access-sw97h\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.634766 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.634682    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-lib-modules\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.736023 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.735934    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-sys\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.736023 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.735979    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-proc\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.736023 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.736002    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-podres\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.736322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.736053    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw97h\" (UniqueName: \"kubernetes.io/projected/4b8a71e6-a72c-4731-b732-5600e9e288b7-kube-api-access-sw97h\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.736322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.736076    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-lib-modules\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.736322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.736092    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-sys\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.736322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.736205    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-podres\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.736322 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.736251    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-proc\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.736547 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.736305    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b8a71e6-a72c-4731-b732-5600e9e288b7-lib-modules\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.750662 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.750629    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw97h\" (UniqueName: \"kubernetes.io/projected/4b8a71e6-a72c-4731-b732-5600e9e288b7-kube-api-access-sw97h\") pod \"perf-node-gather-daemonset-k69mb\" (UID: \"4b8a71e6-a72c-4731-b732-5600e9e288b7\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:26.830390 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:26.830346    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:27.125138 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:27.125051    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"]
Sep 10 21:45:27.129243 ip-10-0-135-74 kubenswrapper[2573]: W0910 21:45:27.129203    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4b8a71e6_a72c_4731_b732_5600e9e288b7.slice/crio-ff2ab3baa1af7a8ca5872668552e083e8610ea20b248e976edad9d460479985a WatchSource:0}: Error finding container ff2ab3baa1af7a8ca5872668552e083e8610ea20b248e976edad9d460479985a: Status 404 returned error can't find the container with id ff2ab3baa1af7a8ca5872668552e083e8610ea20b248e976edad9d460479985a
Sep 10 21:45:27.600899 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:27.600859    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb" event={"ID":"4b8a71e6-a72c-4731-b732-5600e9e288b7","Type":"ContainerStarted","Data":"6d85ab9e09113d0ca0d7c1b125cfe99e3125f3fc72c571a604fe76f4fd683280"}
Sep 10 21:45:27.601344 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:27.600928    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb" event={"ID":"4b8a71e6-a72c-4731-b732-5600e9e288b7","Type":"ContainerStarted","Data":"ff2ab3baa1af7a8ca5872668552e083e8610ea20b248e976edad9d460479985a"}
Sep 10 21:45:27.601759 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:27.601731    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:27.623303 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:27.623243    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb" podStartSLOduration=1.623221503 podStartE2EDuration="1.623221503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:45:27.619770085 +0000 UTC m=+2103.553436897" watchObservedRunningTime="2026-09-10 21:45:27.623221503 +0000 UTC m=+2103.556888320"
Sep 10 21:45:28.943534 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:28.943508    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dqff7_d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8/dns/0.log"
Sep 10 21:45:28.966376 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:28.966347    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dqff7_d5a4a217-d52e-4a0b-8adc-c8d0b4047dc8/kube-rbac-proxy/0.log"
Sep 10 21:45:29.019508 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:29.019482    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dwlgl_cfc5e2be-0dac-4a19-9c82-f70065c9a9b4/dns-node-resolver/0.log"
Sep 10 21:45:29.583833 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:29.583791    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c8bb7b487-s8pmp_f8a26f49-cd50-463a-ae10-38eb845bcbe7/registry/0.log"
Sep 10 21:45:29.658126 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:29.658100    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-svw8v_0e6d31a2-19e9-4970-937a-e30522ffc0df/node-ca/0.log"
Sep 10 21:45:31.077036 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:31.077006    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545c498cb-gc9s9_79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d/kube-auth-proxy/0.log"
Sep 10 21:45:32.096733 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:32.096697    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4mnm5_f701872b-3d4b-4030-9399-56e4fdc01097/serve-healthcheck-canary/0.log"
Sep 10 21:45:32.818643 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:32.818616    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rfmjw_87209e5d-d1b5-4488-bdb0-55fed9ab391c/kube-rbac-proxy/0.log"
Sep 10 21:45:32.841736 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:32.841707    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rfmjw_87209e5d-d1b5-4488-bdb0-55fed9ab391c/exporter/0.log"
Sep 10 21:45:32.864964 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:32.864936    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rfmjw_87209e5d-d1b5-4488-bdb0-55fed9ab391c/extractor/0.log"
Sep 10 21:45:34.624790 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:34.624759    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-k69mb"
Sep 10 21:45:35.544361 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:35.544332    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd_8db3bd51-745a-4ec9-9c39-4145c3a43dac/manager/0.log"
Sep 10 21:45:35.564949 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:35.564920    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-z78jd_8db3bd51-745a-4ec9-9c39-4145c3a43dac/copy-manifests/0.log"
Sep 10 21:45:36.933476 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:36.933447    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66f6f49f76-n7hvf_847954c3-5929-4ea4-a393-31270ca58fb2/manager/0.log"
Sep 10 21:45:37.175344 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:37.175309    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545c498cb-gc9s9_79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d/kube-auth-proxy/0.log"
Sep 10 21:45:39.396058 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:39.395987    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545c498cb-gc9s9_79ac50cb-b3d4-4c21-a0c3-80dc4b58ae4d/kube-auth-proxy/0.log"
Sep 10 21:45:43.805425 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:43.805396    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fc4cj_fa938a91-1541-485a-b88f-151697650d2a/kube-multus-additional-cni-plugins/0.log"
Sep 10 21:45:43.830490 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:43.830451    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fc4cj_fa938a91-1541-485a-b88f-151697650d2a/egress-router-binary-copy/0.log"
Sep 10 21:45:43.852488 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:43.852449    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fc4cj_fa938a91-1541-485a-b88f-151697650d2a/cni-plugins/0.log"
Sep 10 21:45:43.875545 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:43.875488    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fc4cj_fa938a91-1541-485a-b88f-151697650d2a/bond-cni-plugin/0.log"
Sep 10 21:45:43.899263 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:43.899234    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fc4cj_fa938a91-1541-485a-b88f-151697650d2a/routeoverride-cni/0.log"
Sep 10 21:45:43.920364 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:43.920342    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fc4cj_fa938a91-1541-485a-b88f-151697650d2a/whereabouts-cni-bincopy/0.log"
Sep 10 21:45:43.942462 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:43.942435    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fc4cj_fa938a91-1541-485a-b88f-151697650d2a/whereabouts-cni/0.log"
Sep 10 21:45:44.097932 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:44.097900    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wh98p_19e148c2-b280-48d0-ac44-7246e35a71ea/kube-multus/0.log"
Sep 10 21:45:44.216845 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:44.216803    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vqprq_87136fc5-90a6-4728-9410-a51d6050e497/network-metrics-daemon/0.log"
Sep 10 21:45:44.234913 ip-10-0-135-74 kubenswrapper[2573]: I0910 21:45:44.234886    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vqprq_87136fc5-90a6-4728-9410-a51d6050e497/kube-rbac-proxy/0.log"