Jul 26 16:06:39.999594 ip-10-0-135-132 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 26 16:06:39.999609 ip-10-0-135-132 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 26 16:06:39.999619 ip-10-0-135-132 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 26 16:06:39.999925 ip-10-0-135-132 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 26 16:06:50.211641 ip-10-0-135-132 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 26 16:06:50.211658 ip-10-0-135-132 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot db9712079e7d423192399d3198071e7c --
Jul 26 16:08:22.550235 ip-10-0-135-132 systemd[1]: Starting Kubernetes Kubelet...
Jul 26 16:08:22.936412 ip-10-0-135-132 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 16:08:22.936412 ip-10-0-135-132 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 26 16:08:22.936412 ip-10-0-135-132 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 16:08:22.936412 ip-10-0-135-132 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 26 16:08:22.936412 ip-10-0-135-132 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 16:08:22.937074 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.936407    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 26 16:08:22.942336 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942314    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 16:08:22.942336 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942332    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 16:08:22.942336 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942336    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 16:08:22.942336 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942339    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 16:08:22.942336 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942343    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942346    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942349    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942352    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942355    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942357    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942360    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942363    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942366    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942369    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942371    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942374    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942377    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942381    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942385    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942388    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942391    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942394    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942397    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942399    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 16:08:22.942519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942402    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942404    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942409    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942412    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942415    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942418    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942421    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942423    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942426    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942429    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942431    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942434    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942436    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942439    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942443    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942445    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942448    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942451    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942453    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 16:08:22.943049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942456    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942459    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942462    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942465    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942467    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942471    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942475    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942479    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942481    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942484    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942487    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942490    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942493    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942496    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942499    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942501    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942503    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942506    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942508    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942511    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 16:08:22.943527 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942513    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942516    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942518    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942521    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942524    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942527    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942530    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942534    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942536    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942540    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942543    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942545    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942548    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942550    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942553    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942556    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942558    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942561    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942563    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942566    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 16:08:22.944044 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942569    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942572    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942574    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942577    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942579    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942582    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942585    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942587    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942590    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942592    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942595    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942597    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942600    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942602    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942605    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.942608    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944135    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944143    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944147    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 16:08:22.944519 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944150    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944153    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944155    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944159    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944161    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944164    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944169    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944173    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944176    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944179    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944182    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944184    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944187    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944189    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944192    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944196    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944199    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944202    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944205    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 16:08:22.944990 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944209    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944212    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944215    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944218    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944220    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944223    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944226    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944228    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944231    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944234    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944237    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944241    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944245    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944248    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944251    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944254    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944257    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944260    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944263    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 16:08:22.945450 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944266    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944269    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944271    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944274    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944277    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944279    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944282    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944286    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944288    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944293    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944296    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944299    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944303    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944306    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944308    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944311    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944313    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944316    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944319    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 16:08:22.945926 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944321    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944324    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944326    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944329    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944332    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944334    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944337    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944340    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944342    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944345    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944347    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944350    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944352    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944355    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944358    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944361    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944363    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944366    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944368    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944371    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944373    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 16:08:22.946407 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944376    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944378    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944381    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944384    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944386    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944389    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944391    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944394    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944396    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944399    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944401    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944404    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944406    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944409    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944412    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944415    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944417    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944420    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944777    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944782    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 16:08:22.946944 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944784    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944787    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944790    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944792    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944795    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944798    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944801    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944804    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944806    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944809    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944814    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944817    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944820    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944823    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944826    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944829    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944832    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944834    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944837    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944839    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 16:08:22.947425 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944842    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944845    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944847    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944849    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944852    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944854    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944858    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944860    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944863    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944866    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944871    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944874    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944877    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944880    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944883    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944886    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944889    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944892    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944895    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 16:08:22.947976 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944897    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944900    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944903    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944906    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944928    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944931    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944935    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944939    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944943    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944945    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944948    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944951    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944953    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944956    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944959    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944961    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944964    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944967    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944969    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944972    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 16:08:22.948447 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944975    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944977    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944980    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944982    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944985    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944987    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944990    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944993    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944996    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.944998    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945001    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945004    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945007    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945009    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945012    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945014    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945017    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945019    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945022    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945024    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 16:08:22.948951 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945027    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945030    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945032    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945035    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945037    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945040    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945042    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945047    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945049    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945052    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945055    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945058    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945060    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945063    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945065    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945068    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945070    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945073    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945151    2570 flags.go:64] FLAG: --address="0.0.0.0"
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945161    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 26 16:08:22.949463 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945170    2570 flags.go:64] FLAG: --anonymous-auth="true"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945176    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945180    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945183    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945187    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945192    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945195    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945197    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945201    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945205    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945208    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945211    2570 flags.go:64] FLAG: --cgroup-root=""
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945214    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945216    2570 flags.go:64] FLAG: --client-ca-file=""
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945221    2570 flags.go:64] FLAG: --cloud-provider="external"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945224    2570 flags.go:64] FLAG: --cluster-dns="[]"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945228    2570 flags.go:64] FLAG: --cluster-domain=""
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945231    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945234    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945238    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945240    2570 flags.go:64] FLAG: --container-log-max-files="5"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945244    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945247    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 26 16:08:22.949971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945250    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945253    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945256    2570 flags.go:64] FLAG: --contention-profiling="false"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945259    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945262    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945265    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945268    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945272    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945274    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945277    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945280    2570 flags.go:64] FLAG: --enable-load-reader="false"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945283    2570 flags.go:64] FLAG: --enable-server="true"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945286    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945290    2570 flags.go:64] FLAG: --event-burst="100"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945293    2570 flags.go:64] FLAG: --event-qps="50"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945295    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945298    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945301    2570 flags.go:64] FLAG: --eviction-hard=""
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945304    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945307    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945310    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945313    2570 flags.go:64] FLAG: --eviction-soft=""
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945325    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945327    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945331    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 26 16:08:22.950544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945334    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945337    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945340    2570 flags.go:64] FLAG: --fail-swap-on="true"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945343    2570 flags.go:64] FLAG: --feature-gates=""
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945347    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945350    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945353    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945356    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945359    2570 flags.go:64] FLAG: --healthz-port="10248"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945362    2570 flags.go:64] FLAG: --help="false"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945365    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-135-132.ec2.internal"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945368    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945371    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945374    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945378    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945381    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945384    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945387    2570 flags.go:64] FLAG: --image-service-endpoint=""
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945390    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945392    2570 flags.go:64] FLAG: --kube-api-burst="100"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945395    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945398    2570 flags.go:64] FLAG: --kube-api-qps="50"
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945401    2570 flags.go:64] FLAG: --kube-reserved=""
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945404    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 26 16:08:22.951250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945406    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945409    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945413    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945415    2570 flags.go:64] FLAG: --lock-file=""
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945418    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945421    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945424    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945429    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945432    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945438    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945441    2570 flags.go:64] FLAG: --logging-format="text"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945444    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945447    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945452    2570 flags.go:64] FLAG: --manifest-url=""
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945455    2570 flags.go:64] FLAG: --manifest-url-header=""
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945459    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945462    2570 flags.go:64] FLAG: --max-open-files="1000000"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945466    2570 flags.go:64] FLAG: --max-pods="110"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945469    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945472    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945475    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945478    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945481    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945484    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945487    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 26 16:08:22.951880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945494    2570 flags.go:64] FLAG: --node-status-max-images="50"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945497    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945500    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945503    2570 flags.go:64] FLAG: --pod-cidr=""
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945506    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945512    2570 flags.go:64] FLAG: --pod-manifest-path=""
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945515    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945518    2570 flags.go:64] FLAG: --pods-per-core="0"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945520    2570 flags.go:64] FLAG: --port="10250"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945524    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945527    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02be537040dc977b7"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945530    2570 flags.go:64] FLAG: --qos-reserved=""
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945533    2570 flags.go:64] FLAG: --read-only-port="10255"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945536    2570 flags.go:64] FLAG: --register-node="true"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945538    2570 flags.go:64] FLAG: --register-with-taints=""
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945542    2570 flags.go:64] FLAG: --registry-burst="10"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945547    2570 flags.go:64] FLAG: --registry-qps="5"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945550    2570 flags.go:64] FLAG: --reserved-cpus=""
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945552    2570 flags.go:64] FLAG: --reserved-memory=""
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945561    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945564    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945568    2570 flags.go:64] FLAG: --rotate-certificates="false"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945571    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945574    2570 flags.go:64] FLAG: --runonce="false"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945577    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 26 16:08:22.952519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945580    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945583    2570 flags.go:64] FLAG: --seccomp-default="false"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945586    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945589    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945592    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945595    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945598    2570 flags.go:64] FLAG: --storage-driver-password="root"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945601    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945603    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945606    2570 flags.go:64] FLAG: --storage-driver-user="root"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945609    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945612    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945615    2570 flags.go:64] FLAG: --system-cgroups=""
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945618    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945623    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945626    2570 flags.go:64] FLAG: --tls-cert-file=""
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945628    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945633    2570 flags.go:64] FLAG: --tls-min-version=""
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945635    2570 flags.go:64] FLAG: --tls-private-key-file=""
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945638    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945641    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945644    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945647    2570 flags.go:64] FLAG: --v="2"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945652    2570 flags.go:64] FLAG: --version="false"
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945656    2570 flags.go:64] FLAG: --vmodule=""
Jul 26 16:08:22.953187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945660    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.945663    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945772    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945777    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945780    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945783    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945786    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945788    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945792    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945796    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945799    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945802    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945805    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945808    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945810    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945813    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945815    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945818    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945821    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945823    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 16:08:22.953799 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945826    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945828    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945831    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945834    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945836    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945839    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945842    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945844    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945847    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945849    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945855    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945858    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945861    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945864    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945868    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945873    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945876    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945878    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945881    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945884    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 16:08:22.954320 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945887    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945889    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945892    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945894    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945897    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945899    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945902    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945905    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945920    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945924    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945928    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945933    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945936    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945938    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945941    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945944    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945946    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945949    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945951    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945954    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 16:08:22.954808 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945956    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945959    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945963    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945965    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945968    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945971    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945975    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945980    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945983    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945985    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945988    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945991    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945993    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945996    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.945998    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946001    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946004    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946006    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946009    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946011    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 16:08:22.955301 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946014    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946017    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946019    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946022    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946024    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946027    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946030    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946032    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946035    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946037    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946040    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946042    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946045    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946047    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946052    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946054    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946057    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946059    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946062    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946067    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 16:08:22.955805 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.946069    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.946946    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.953670    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.953685    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.954532    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.954543    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954599    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954604    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954607    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954610    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954612    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954615    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954620    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954624    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 16:08:22.956296 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954627    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954630    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954633    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954635    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954638    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954640    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954643    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954646    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954649    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954651    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954654    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954656    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954659    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954661    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954664    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954666    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954669    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954671    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954674    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954676    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 16:08:22.956646 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954679    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954683    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954686    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954688    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954691    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954693    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954696    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954698    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954701    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954703    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954706    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954708    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954711    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954713    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954715    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954718    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954720    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954723    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954725    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954728    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 16:08:22.957147 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954730    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954733    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954735    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954738    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954740    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954743    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954745    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954748    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954750    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954753    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954755    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954758    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954760    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954765    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954770    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954773    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954776    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954779    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954782    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 16:08:22.957652 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954784    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954787    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954789    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954792    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954795    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954798    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954800    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954803    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954805    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954808    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954811    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954813    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954816    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954819    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954821    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954824    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954827    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954829    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954833    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954836    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 16:08:22.958138 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954839    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954842    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954844    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954847    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954850    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954852    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954855    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954858    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954861    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954864    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954866    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.954869    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.954874    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955014    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955020    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 16:08:22.958607 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955023    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955025    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955028    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955031    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955033    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955036    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955038    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955041    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955043    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955046    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955049    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955051    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955054    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955056    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955058    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955061    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955064    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955067    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955069    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955072    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 16:08:22.958989 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955074    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955077    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955080    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955083    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955086    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955088    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955091    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955094    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955096    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955099    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955102    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955104    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955106    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955109    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955111    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955114    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955116    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955119    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955121    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955124    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 16:08:22.959478 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955126    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955129    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955132    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955134    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955137    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955139    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955142    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955144    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955146    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955149    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955151    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955154    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955156    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955158    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955162    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955164    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955167    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955169    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955172    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955175    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 16:08:22.960123 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955177    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955180    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955182    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955184    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955187    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955189    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955192    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955194    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955197    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955200    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955202    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955204    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955207    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955209    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955213    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955216    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955219    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955222    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955224    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 16:08:22.960591 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955227    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955231    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955234    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955238    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955240    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955243    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955245    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955248    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955252    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955256    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955259    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955262    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955265    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955268    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955271    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955274    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955277    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 16:08:22.961073 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:22.955280    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 16:08:22.961475 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.955284    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 16:08:22.961475 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.955380    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 26 16:08:22.961475 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.957249    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 26 16:08:22.961475 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.958411    2570 server.go:1019] "Starting client certificate rotation"
Jul 26 16:08:22.961475 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.958510    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 16:08:22.961475 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.958547    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 16:08:22.961475 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.960691    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 16:08:22.962031 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.961998    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 16:08:22.974769 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.974752    2570 log.go:25] "Validated CRI v1 runtime API"
Jul 26 16:08:22.977431 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.977414    2570 log.go:25] "Validated CRI v1 image API"
Jul 26 16:08:22.978234 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.978218    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 26 16:08:22.980766 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.980748    2570 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9dbf5835-4ee1-4b37-a396-213b099be00d:/dev/nvme0n1p3 b6bf2bf9-1bf8-4aa8-9950-460b8cd3da20:/dev/nvme0n1p4]
Jul 26 16:08:22.980827 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.980766    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 26 16:08:22.985466 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.985448    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 16:08:22.986742 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.986637    2570 manager.go:216] Machine: {Timestamp:2026-07-26 16:08:22.985243605 +0000 UTC m=+0.319604747 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101042 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec241d0fac80e3a701fbec8ab48ab78a SystemUUID:ec241d0f-ac80-e3a7-01fb-ec8ab48ab78a BootID:db971207-9e7d-4231-9239-9d3198071e7c Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4c:cb:36:b9:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4c:cb:36:b9:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:d5:15:1e:67:46 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 26 16:08:22.986742 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.986742    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 26 16:08:22.986867 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.986816    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 26 16:08:22.989540 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.989518    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 26 16:08:22.989671 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.989542    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-132.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 26 16:08:22.990215 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.990205    2570 topology_manager.go:138] "Creating topology manager with none policy"
Jul 26 16:08:22.990254 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.990216    2570 container_manager_linux.go:309] "Creating device plugin manager"
Jul 26 16:08:22.990254 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.990224    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 16:08:22.990987 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.990977    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 16:08:22.991790 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.991780    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 26 16:08:22.994358 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.994347    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 16:08:22.994472 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.994463    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 26 16:08:22.994507 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.994492    2570 kubelet.go:486] "Attempting to sync node with API server"
Jul 26 16:08:22.994538 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.994508    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 26 16:08:22.994538 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.994519    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 26 16:08:22.994538 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.994527    2570 kubelet.go:398] "Adding apiserver pod source"
Jul 26 16:08:22.994538 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.994535    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 26 16:08:22.994949 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.994935    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 16:08:22.995017 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.994957    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 16:08:22.997160 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.997139    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 26 16:08:22.999041 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.999027    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 26 16:08:22.999108 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:22.999057    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 26 16:08:23.001089 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001076    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 26 16:08:23.001170 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001098    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 26 16:08:23.001170 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001106    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 26 16:08:23.001170 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001113    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 26 16:08:23.001170 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001125    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 26 16:08:23.001170 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001135    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 26 16:08:23.001170 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001160    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 26 16:08:23.001170 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001166    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 26 16:08:23.001392 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001177    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 26 16:08:23.001392 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001183    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 26 16:08:23.001392 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001200    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 26 16:08:23.001392 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001206    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 26 16:08:23.001999 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001989    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 26 16:08:23.001999 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.001999    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 26 16:08:23.005139 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.005126    2570 server.go:1268] "Started kubelet"
Jul 26 16:08:23.005225 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.005202    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 26 16:08:23.005302 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.005216    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 26 16:08:23.005357 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.005343    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 26 16:08:23.005438 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.005422    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-132.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 26 16:08:23.005597 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.005536    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 26 16:08:23.005660 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.005620    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-132.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 26 16:08:23.006049 ip-10-0-135-132 systemd[1]: Started Kubernetes Kubelet.
Jul 26 16:08:23.007085 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.007068    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 26 16:08:23.010152 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.010084    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-92vp5"
Jul 26 16:08:23.010797 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.010777    2570 server.go:310] "Adding debug handlers to kubelet server"
Jul 26 16:08:23.011276 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.011250    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 26 16:08:23.015550 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.015532    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 26 16:08:23.015656 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.015557    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 26 16:08:23.015727 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.015664    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found"
Jul 26 16:08:23.015814 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.015794    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-92vp5"
Jul 26 16:08:23.016077 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.016060    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 26 16:08:23.016077 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.016078    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 26 16:08:23.016210 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.016107    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 26 16:08:23.016210 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.016195    2570 reconstruct.go:97] "Volume reconstruction finished"
Jul 26 16:08:23.016210 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.016203    2570 reconciler.go:29] "Reconciler: start to sync state"
Jul 26 16:08:23.016428 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.015502    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-132.ec2.internal.18c5e2b70486bac4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-132.ec2.internal,UID:ip-10-0-135-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-132.ec2.internal,},FirstTimestamp:2026-07-26 16:08:23.00510074 +0000 UTC m=+0.339461881,LastTimestamp:2026-07-26 16:08:23.00510074 +0000 UTC m=+0.339461881,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-132.ec2.internal,}"
Jul 26 16:08:23.016871 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.016857    2570 factory.go:55] Registering systemd factory
Jul 26 16:08:23.016966 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.016874    2570 factory.go:223] Registration of the systemd container factory successfully
Jul 26 16:08:23.017119 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.017106    2570 factory.go:153] Registering CRI-O factory
Jul 26 16:08:23.017156 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.017121    2570 factory.go:223] Registration of the crio container factory successfully
Jul 26 16:08:23.017184 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.017171    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 26 16:08:23.017230 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.017192    2570 factory.go:103] Registering Raw factory
Jul 26 16:08:23.017230 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.017206    2570 manager.go:1196] Started watching for new ooms in manager
Jul 26 16:08:23.017644 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.017630    2570 manager.go:349] Starting recovery of all containers
Jul 26 16:08:23.022963 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.022937    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-132.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 26 16:08:23.023087 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.023066    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 26 16:08:23.028091 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.028066    2570 manager.go:354] Recovery completed
Jul 26 16:08:23.034886 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.034871    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 16:08:23.036752 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.036735    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 16:08:23.036847 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.036774    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 16:08:23.036847 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.036788    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID"
Jul 26 16:08:23.037351 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.037337    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 26 16:08:23.037351 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.037350    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 26 16:08:23.037480 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.037362    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 16:08:23.037480 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.037368    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 16:08:23.038132 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.038115    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 16:08:23.038213 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.038141    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 16:08:23.038213 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.038153    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID"
Jul 26 16:08:23.038752 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.038682    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-132.ec2.internal.18c5e2b70669c995  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-132.ec2.internal,UID:ip-10-0-135-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-132.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-132.ec2.internal,},FirstTimestamp:2026-07-26 16:08:23.036758421 +0000 UTC m=+0.371119563,LastTimestamp:2026-07-26 16:08:23.036758421 +0000 UTC m=+0.371119563,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-132.ec2.internal,}"
Jul 26 16:08:23.040132 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.040118    2570 policy_none.go:49] "None policy: Start"
Jul 26 16:08:23.040194 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.040136    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 26 16:08:23.040194 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.040146    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 26 16:08:23.042090 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.042079    2570 policy_none.go:47] "Start"
Jul 26 16:08:23.081529 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.081513    2570 manager.go:337] "Starting Device Plugin manager"
Jul 26 16:08:23.081642 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.081551    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 26 16:08:23.081642 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.081561    2570 server.go:87] "Starting device plugin registration server"
Jul 26 16:08:23.081852 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.081839    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 26 16:08:23.081924 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.081854    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 26 16:08:23.082064 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.081968    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 26 16:08:23.082154 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.082064    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 26 16:08:23.082154 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.082075    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 26 16:08:23.082579 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.082561    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 26 16:08:23.082652 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.082611    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-132.ec2.internal\" not found"
Jul 26 16:08:23.094028 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.094011    2570 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2604 into the system container \"/system.slice\": <nil>"
Jul 26 16:08:23.141348 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.141322    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 26 16:08:23.143526 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.142472    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 26 16:08:23.143526 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.142517    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 26 16:08:23.143526 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.142541    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 26 16:08:23.143526 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.142583    2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 26 16:08:23.144579 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.144559    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 16:08:23.182389 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.182348    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 16:08:23.183304 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.183283    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 16:08:23.183410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.183320    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 16:08:23.183410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.183336    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID"
Jul 26 16:08:23.183410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.183361    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.191428 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.191387    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.191428 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.191408    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-132.ec2.internal\": node \"ip-10-0-135-132.ec2.internal\" not found"
Jul 26 16:08:23.205009 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.204994    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found"
Jul 26 16:08:23.243316 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.243160    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal"]
Jul 26 16:08:23.243400 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.243350    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 16:08:23.244205 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.244191    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 16:08:23.244282 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.244219    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 16:08:23.244282 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.244229    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID"
Jul 26 16:08:23.246433 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.246420    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 16:08:23.246555 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.246540    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.246616 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.246588    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 16:08:23.247094 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.247076    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 16:08:23.247187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.247107    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 16:08:23.247187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.247076    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 16:08:23.247187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.247121    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID"
Jul 26 16:08:23.247187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.247135    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 16:08:23.247187 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.247150    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID"
Jul 26 16:08:23.249338 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.249323    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.249421 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.249354    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 16:08:23.250069 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.250050    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 16:08:23.250155 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.250081    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 16:08:23.250155 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.250096    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID"
Jul 26 16:08:23.267496 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.267473    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-132.ec2.internal\" not found" node="ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.270851 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.270830    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-132.ec2.internal\" not found" node="ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.306050 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.306027    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found"
Jul 26 16:08:23.317642 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.317616    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a3de1b13181d39474231213cb35e8b2d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-132.ec2.internal\" (UID: \"a3de1b13181d39474231213cb35e8b2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.317743 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.317646    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31b67110ee781749d3f159efc4905833-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"31b67110ee781749d3f159efc4905833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.317743 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.317673    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31b67110ee781749d3f159efc4905833-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"31b67110ee781749d3f159efc4905833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.406295 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.406264    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found"
Jul 26 16:08:23.418604 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.418581    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a3de1b13181d39474231213cb35e8b2d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-132.ec2.internal\" (UID: \"a3de1b13181d39474231213cb35e8b2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.418687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.418612    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31b67110ee781749d3f159efc4905833-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"31b67110ee781749d3f159efc4905833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.418687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.418637    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31b67110ee781749d3f159efc4905833-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"31b67110ee781749d3f159efc4905833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.418787 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.418717    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a3de1b13181d39474231213cb35e8b2d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-132.ec2.internal\" (UID: \"a3de1b13181d39474231213cb35e8b2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.418787 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.418758    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31b67110ee781749d3f159efc4905833-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"31b67110ee781749d3f159efc4905833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.418857 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.418799    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31b67110ee781749d3f159efc4905833-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"31b67110ee781749d3f159efc4905833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.507108 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.507028    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found"
Jul 26 16:08:23.570548 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.570522    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.573646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.573629    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.607603 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.607561    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found"
Jul 26 16:08:23.708086 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.708055    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found"
Jul 26 16:08:23.808618 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:23.808546    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found"
Jul 26 16:08:23.848101 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.848079    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 16:08:23.915980 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.915954    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.926313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.926280    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 16:08:23.928139 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.928123    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal"
Jul 26 16:08:23.937770 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.937752    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 16:08:23.957922 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.957892    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 26 16:08:23.958031 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.958010    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 16:08:23.958076 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.958047    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 16:08:23.976091 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.976069    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 16:08:23.995476 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:23.995453    2570 apiserver.go:52] "Watching apiserver"
Jul 26 16:08:24.006153 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.006134    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 26 16:08:24.007119 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.007100    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-nb5cd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal","openshift-multus/multus-5w646","openshift-cluster-node-tuning-operator/tuned-t79zb","openshift-multus/multus-additional-cni-plugins-wb6b9","openshift-multus/network-metrics-daemon-7xxbv","openshift-network-diagnostics/network-check-target-djgt6","openshift-network-operator/iptables-alerter-b59qf","openshift-ovn-kubernetes/ovnkube-node-dn7k8","kube-system/global-pull-secret-syncer-5dfgr","kube-system/konnectivity-agent-lxsxf","kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26","openshift-dns/node-resolver-6f5mx"]
Jul 26 16:08:24.009492 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.009472    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:24.009569 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:24.009542    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:24.011881 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.011863    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:24.015055 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.015033    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 26 16:08:24.015276 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.015253    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vltp9\""
Jul 26 16:08:24.015276 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.015041    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 26 16:08:24.015462 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.015371    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 26 16:08:24.015847 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.015820    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 26 16:08:24.016072 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.016056    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.018245 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.018228    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 26 16:08:24.018518 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.018502    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.018598 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.018541    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 26 16:08:24.018661 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.018640    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 26 16:08:24.018740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.018722    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 26 16:08:24.018870 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.018857    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w2j8x\""
Jul 26 16:08:24.020809 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.020792    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ptq2r\""
Jul 26 16:08:24.020809 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.020797    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 26 16:08:24.021080 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021065    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 26 16:08:24.021170 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021155    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:24.021236 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021222    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-multus-conf-dir\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.021273 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-modprobe-d\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.021318 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021274    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-sysctl-d\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.021318 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021297    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-sysctl-conf\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.021387 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021324    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-run\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.021387 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021347    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-var-lib-kubelet\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.021387 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021369    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-etc-selinux\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:24.021487 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021392    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6954dc32-3e67-4813-9eb7-ea925571876e-cni-binary-copy\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.021487 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021412    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-run-netns\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.021487 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021426    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-var-lib-cni-multus\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.021487 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021442    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-systemd\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.021487 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021464    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-run-k8s-cni-cncf-io\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.021487 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021482    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-hostroot\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.021744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021498    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kszzk\" (UniqueName: \"kubernetes.io/projected/8bafccda-1228-489e-a79b-88d0d1455a82-kube-api-access-kszzk\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:24.021744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021516    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-os-release\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.021744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021556    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-socket-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:24.021744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021587    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-sys-fs\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:24.021744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021622    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-var-lib-kubelet\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.021744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021646    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-sysconfig\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.021744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021667    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-kubernetes\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.021744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021691    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-multus-cni-dir\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.021744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021712    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-etc-kubernetes\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.021744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021733    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-host\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021756    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66b5a38b-aed2-446f-8f8e-615b1f9557e7-tmp\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021778    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-run-multus-certs\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021801    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-cnibin\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021825    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-registration-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021848    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-device-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021878    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-system-cni-dir\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021900    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6954dc32-3e67-4813-9eb7-ea925571876e-multus-daemon-config\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021947    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-lib-modules\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.021971    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-tuned\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.022016    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdb5s\" (UniqueName: \"kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s\") pod \"network-check-target-djgt6\" (UID: \"f7f5558e-066e-4005-9d86-3090d19d3d66\") " pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.022041    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-multus-socket-dir-parent\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.022070    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-var-lib-cni-bin\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.022092    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpqpb\" (UniqueName: \"kubernetes.io/projected/6954dc32-3e67-4813-9eb7-ea925571876e-kube-api-access-hpqpb\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.022106    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-sys\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.022128    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kcfw\" (UniqueName: \"kubernetes.io/projected/66b5a38b-aed2-446f-8f8e-615b1f9557e7-kube-api-access-4kcfw\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:24.022178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.022156    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:24.023604 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.023587    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 26 16:08:24.023684 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.023590    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qv8fl\""
Jul 26 16:08:24.023745 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.023691    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 26 16:08:24.023860 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.023844    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:24.023954 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:24.023932    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:24.026498 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.026464    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-25 16:03:24 +0000 UTC" deadline="2028-03-17 16:36:55.694361631 +0000 UTC"
Jul 26 16:08:24.026498 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.026496    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14400h28m31.667867701s"
Jul 26 16:08:24.026616 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.026512    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:24.029145 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.029129    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 26 16:08:24.029244 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.029197    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 26 16:08:24.029306 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.029272    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 16:08:24.029306 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:24.029291    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.181953 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.181617    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hq5gc\""
Jul 26 16:08:25.181953 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.181780    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 26 16:08:25.181953 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.181792    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 26 16:08:25.181953 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.181787    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 26 16:08:25.181953 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.181783    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 26 16:08:25.182852 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.182837    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 26 16:08:25.182936 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.182921    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 26 16:08:25.183147 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.183132    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 26 16:08:25.183245 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.183170    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.183245 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.183199    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dkfmz\""
Jul 26 16:08:25.185549 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.185531    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 26 16:08:25.185646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.185593    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 26 16:08:25.185754 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.185740    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9gdj6\""
Jul 26 16:08:25.185864 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.185849    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:25.185934 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.185916    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:25.188465 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.188446    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:25.191101 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.191070    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 26 16:08:25.191294 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.191113    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 26 16:08:25.191294 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.191224    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.191294 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.191283    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bcs97\""
Jul 26 16:08:25.193646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.193628    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 26 16:08:25.193828 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.193815    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 26 16:08:25.194048 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.194036    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jdg84\""
Jul 26 16:08:25.194183 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.194172    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 26 16:08:25.200043 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.200024    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vvr79"
Jul 26 16:08:25.207279 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.207258    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vvr79"
Jul 26 16:08:25.265927 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.265899    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 16:08:25.268014 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.267996    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 26 16:08:25.273840 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.273817    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.273971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.273851    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-sysctl-conf\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.273971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.273877    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-run\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.273971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.273890    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.273971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.273929    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.273971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.273953    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-ovn-node-metrics-cert\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.273976    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/74387622-269d-4e67-af29-776fabd1e176-agent-certs\") pod \"konnectivity-agent-lxsxf\" (UID: \"74387622-269d-4e67-af29-776fabd1e176\") " pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.273988    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-sysctl-conf\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.273996    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-run\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274004    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6954dc32-3e67-4813-9eb7-ea925571876e-cni-binary-copy\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274028    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-os-release\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274049    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-hostroot\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274064    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-run-openvswitch\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274078    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-node-log\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274100    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-cni-netd\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274123    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-ovnkube-config\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274125    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-hostroot\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274167    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b5k8\" (UniqueName: \"kubernetes.io/projected/eca06d87-f10e-47d5-bb84-05d33e21d3c6-kube-api-access-4b5k8\") pod \"iptables-alerter-b59qf\" (UID: \"eca06d87-f10e-47d5-bb84-05d33e21d3c6\") " pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274224    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszzk\" (UniqueName: \"kubernetes.io/projected/8bafccda-1228-489e-a79b-88d0d1455a82-kube-api-access-kszzk\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274252    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-socket-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274274    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-var-lib-kubelet\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274300    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-run-systemd\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274323    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-run-ovn\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274369    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-var-lib-kubelet\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274345    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15961897-ec82-41cf-8ce5-b036a30b987c-kubelet-config\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274392    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-socket-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274420    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-system-cni-dir\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274455    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01f4e8e6-8355-42ec-9f7a-33c4f0d68c90-tmp-dir\") pod \"node-resolver-6f5mx\" (UID: \"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90\") " pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274481    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-etc-kubernetes\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274516    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-etc-kubernetes\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274529    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6954dc32-3e67-4813-9eb7-ea925571876e-cni-binary-copy\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274542    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-log-socket\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274567    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-cni-bin\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274592    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqmk5\" (UniqueName: \"kubernetes.io/projected/e0b75180-bc1f-42fa-9792-c2de53a22d56-kube-api-access-wqmk5\") pod \"node-ca-nb5cd\" (UID: \"e0b75180-bc1f-42fa-9792-c2de53a22d56\") " pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274647    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-run-multus-certs\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274698    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-cnibin\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274705    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-run-multus-certs\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274748    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-cnibin\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274755    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6954dc32-3e67-4813-9eb7-ea925571876e-multus-daemon-config\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.274993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274797    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-lib-modules\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274824    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-cnibin\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274851    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdb5s\" (UniqueName: \"kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s\") pod \"network-check-target-djgt6\" (UID: \"f7f5558e-066e-4005-9d86-3090d19d3d66\") " pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274874    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-sys\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274897    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-multus-conf-dir\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274915    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-lib-modules\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274966    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-multus-conf-dir\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274960    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-sys\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.274919    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-modprobe-d\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275014    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-modprobe-d\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275025    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-sysctl-d\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275053    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-var-lib-kubelet\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275076    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-etc-selinux\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275099    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-run-netns\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275125    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-var-lib-cni-multus\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275142    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-sysctl-d\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275150    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-systemd\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275165    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6954dc32-3e67-4813-9eb7-ea925571876e-multus-daemon-config\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.275861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275201    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-slash\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275207    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-systemd\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275226    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsq9b\" (UniqueName: \"kubernetes.io/projected/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-kube-api-access-vsq9b\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275236    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-etc-selinux\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275244    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bf939ed-2d88-4258-9694-e2b0a924478c-cni-binary-copy\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275248    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-run-netns\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275294    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275308    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-var-lib-cni-multus\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275316    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-var-lib-kubelet\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275360    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-run-k8s-cni-cncf-io\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275386    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsmq7\" (UniqueName: \"kubernetes.io/projected/9d8d82fd-3ead-4428-896e-103c296da485-kube-api-access-wsmq7\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275427    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-run-k8s-cni-cncf-io\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275437    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-kubelet\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275475    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0b75180-bc1f-42fa-9792-c2de53a22d56-host\") pod \"node-ca-nb5cd\" (UID: \"e0b75180-bc1f-42fa-9792-c2de53a22d56\") " pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275508    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275551    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/74387622-269d-4e67-af29-776fabd1e176-konnectivity-ca\") pod \"konnectivity-agent-lxsxf\" (UID: \"74387622-269d-4e67-af29-776fabd1e176\") " pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275588    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-os-release\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.276717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275612    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bf939ed-2d88-4258-9694-e2b0a924478c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275640    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb7mc\" (UniqueName: \"kubernetes.io/projected/5bf939ed-2d88-4258-9694-e2b0a924478c-kube-api-access-jb7mc\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275662    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01f4e8e6-8355-42ec-9f7a-33c4f0d68c90-hosts-file\") pod \"node-resolver-6f5mx\" (UID: \"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90\") " pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275677    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-os-release\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275686    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-sys-fs\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275708    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-sysconfig\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275759    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-sysconfig\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275767    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-sys-fs\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275768    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-kubernetes\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275793    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-systemd-units\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275814    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-var-lib-openvswitch\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275819    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-kubernetes\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275835    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15961897-ec82-41cf-8ce5-b036a30b987c-dbus\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275854    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eca06d87-f10e-47d5-bb84-05d33e21d3c6-iptables-alerter-script\") pod \"iptables-alerter-b59qf\" (UID: \"eca06d87-f10e-47d5-bb84-05d33e21d3c6\") " pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275878    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-run-ovn-kubernetes\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275905    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-multus-cni-dir\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275935    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-host\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.277259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275966    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66b5a38b-aed2-446f-8f8e-615b1f9557e7-tmp\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.275988    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276009    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-etc-openvswitch\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276039    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-env-overrides\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276045    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-multus-cni-dir\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276062    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-ovnkube-script-lib\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276113    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kcfw\" (UniqueName: \"kubernetes.io/projected/66b5a38b-aed2-446f-8f8e-615b1f9557e7-kube-api-access-4kcfw\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276115    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66b5a38b-aed2-446f-8f8e-615b1f9557e7-host\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276143    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-registration-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276249    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276263    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-device-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276281    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-registration-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276313    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-system-cni-dir\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276334    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bafccda-1228-489e-a79b-88d0d1455a82-device-dir\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276344    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-system-cni-dir\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276380    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-tuned\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276405    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0b75180-bc1f-42fa-9792-c2de53a22d56-serviceca\") pod \"node-ca-nb5cd\" (UID: \"e0b75180-bc1f-42fa-9792-c2de53a22d56\") " pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.277740 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276420    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bf939ed-2d88-4258-9694-e2b0a924478c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.278215 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276455    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eca06d87-f10e-47d5-bb84-05d33e21d3c6-host-slash\") pod \"iptables-alerter-b59qf\" (UID: \"eca06d87-f10e-47d5-bb84-05d33e21d3c6\") " pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.278215 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276490    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-multus-socket-dir-parent\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.278215 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276514    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-var-lib-cni-bin\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.278215 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276539    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpqpb\" (UniqueName: \"kubernetes.io/projected/6954dc32-3e67-4813-9eb7-ea925571876e-kube-api-access-hpqpb\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.278215 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276563    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-run-netns\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.278215 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276588    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77kph\" (UniqueName: \"kubernetes.io/projected/01f4e8e6-8355-42ec-9f7a-33c4f0d68c90-kube-api-access-77kph\") pod \"node-resolver-6f5mx\" (UID: \"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90\") " pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.278215 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276609    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-host-var-lib-cni-bin\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.278215 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.276630    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6954dc32-3e67-4813-9eb7-ea925571876e-multus-socket-dir-parent\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.279159 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.279145    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66b5a38b-aed2-446f-8f8e-615b1f9557e7-tmp\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.279234 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.279175    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66b5a38b-aed2-446f-8f8e-615b1f9557e7-etc-tuned\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.280594 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.280573    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 16:08:25.280652 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.280595    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 16:08:25.280652 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.280606    2570 projected.go:196] Error preparing data for projected volume kube-api-access-tdb5s for pod openshift-network-diagnostics/network-check-target-djgt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:25.280730 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.280664    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s podName:f7f5558e-066e-4005-9d86-3090d19d3d66 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:25.780641028 +0000 UTC m=+1.963725293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tdb5s" (UniqueName: "kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s") pod "network-check-target-djgt6" (UID: "f7f5558e-066e-4005-9d86-3090d19d3d66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:25.282503 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.282482    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kszzk\" (UniqueName: \"kubernetes.io/projected/8bafccda-1228-489e-a79b-88d0d1455a82-kube-api-access-kszzk\") pod \"aws-ebs-csi-driver-node-nzz26\" (UID: \"8bafccda-1228-489e-a79b-88d0d1455a82\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.283280 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.283258    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kcfw\" (UniqueName: \"kubernetes.io/projected/66b5a38b-aed2-446f-8f8e-615b1f9557e7-kube-api-access-4kcfw\") pod \"tuned-t79zb\" (UID: \"66b5a38b-aed2-446f-8f8e-615b1f9557e7\") " pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.283825 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.283810    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpqpb\" (UniqueName: \"kubernetes.io/projected/6954dc32-3e67-4813-9eb7-ea925571876e-kube-api-access-hpqpb\") pod \"multus-5w646\" (UID: \"6954dc32-3e67-4813-9eb7-ea925571876e\") " pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.335595 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.335511    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3de1b13181d39474231213cb35e8b2d.slice/crio-16e7eed5cd07ae5f8e57e5a754eade4518d00480c21a426e0980ed372c5e5398 WatchSource:0}: Error finding container 16e7eed5cd07ae5f8e57e5a754eade4518d00480c21a426e0980ed372c5e5398: Status 404 returned error can't find the container with id 16e7eed5cd07ae5f8e57e5a754eade4518d00480c21a426e0980ed372c5e5398
Jul 26 16:08:25.340897 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.340877    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:08:25.346950 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.346926    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31b67110ee781749d3f159efc4905833.slice/crio-6471ba0e2e640cd2981d2408726a2616b2c28eaf77f05382858d1cd55a8d8b35 WatchSource:0}: Error finding container 6471ba0e2e640cd2981d2408726a2616b2c28eaf77f05382858d1cd55a8d8b35: Status 404 returned error can't find the container with id 6471ba0e2e640cd2981d2408726a2616b2c28eaf77f05382858d1cd55a8d8b35
Jul 26 16:08:25.377009 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.376986    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bf939ed-2d88-4258-9694-e2b0a924478c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.377123 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377021    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb7mc\" (UniqueName: \"kubernetes.io/projected/5bf939ed-2d88-4258-9694-e2b0a924478c-kube-api-access-jb7mc\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.377123 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377045    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01f4e8e6-8355-42ec-9f7a-33c4f0d68c90-hosts-file\") pod \"node-resolver-6f5mx\" (UID: \"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90\") " pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.377123 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377109    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01f4e8e6-8355-42ec-9f7a-33c4f0d68c90-hosts-file\") pod \"node-resolver-6f5mx\" (UID: \"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90\") " pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.377267 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377162    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-systemd-units\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377267 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377215    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-var-lib-openvswitch\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377267 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377242    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15961897-ec82-41cf-8ce5-b036a30b987c-dbus\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:25.377410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377268    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eca06d87-f10e-47d5-bb84-05d33e21d3c6-iptables-alerter-script\") pod \"iptables-alerter-b59qf\" (UID: \"eca06d87-f10e-47d5-bb84-05d33e21d3c6\") " pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.377410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377284    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-systemd-units\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377294    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-run-ovn-kubernetes\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377314    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-var-lib-openvswitch\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377322    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:25.377410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377342    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15961897-ec82-41cf-8ce5-b036a30b987c-dbus\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:25.377410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377356    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-etc-openvswitch\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377390    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-env-overrides\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377410 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.377400    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:25.377410 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377407    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-run-ovn-kubernetes\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377413    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-ovnkube-script-lib\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377397    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-etc-openvswitch\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.377456    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs podName:9d8d82fd-3ead-4428-896e-103c296da485 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:25.87743634 +0000 UTC m=+2.060520641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs") pod "network-metrics-daemon-7xxbv" (UID: "9d8d82fd-3ead-4428-896e-103c296da485") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377511    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0b75180-bc1f-42fa-9792-c2de53a22d56-serviceca\") pod \"node-ca-nb5cd\" (UID: \"e0b75180-bc1f-42fa-9792-c2de53a22d56\") " pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377538    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bf939ed-2d88-4258-9694-e2b0a924478c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377565    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eca06d87-f10e-47d5-bb84-05d33e21d3c6-host-slash\") pod \"iptables-alerter-b59qf\" (UID: \"eca06d87-f10e-47d5-bb84-05d33e21d3c6\") " pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377591    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-run-netns\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377617    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77kph\" (UniqueName: \"kubernetes.io/projected/01f4e8e6-8355-42ec-9f7a-33c4f0d68c90-kube-api-access-77kph\") pod \"node-resolver-6f5mx\" (UID: \"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90\") " pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377633    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bf939ed-2d88-4258-9694-e2b0a924478c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377646    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377670    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-ovn-node-metrics-cert\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377685    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eca06d87-f10e-47d5-bb84-05d33e21d3c6-host-slash\") pod \"iptables-alerter-b59qf\" (UID: \"eca06d87-f10e-47d5-bb84-05d33e21d3c6\") " pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377693    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/74387622-269d-4e67-af29-776fabd1e176-agent-certs\") pod \"konnectivity-agent-lxsxf\" (UID: \"74387622-269d-4e67-af29-776fabd1e176\") " pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377719    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-os-release\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377744    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-run-openvswitch\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.377862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377768    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-node-log\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377790    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-cni-netd\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377818    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-ovnkube-config\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377843    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b5k8\" (UniqueName: \"kubernetes.io/projected/eca06d87-f10e-47d5-bb84-05d33e21d3c6-kube-api-access-4b5k8\") pod \"iptables-alerter-b59qf\" (UID: \"eca06d87-f10e-47d5-bb84-05d33e21d3c6\") " pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377858    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eca06d87-f10e-47d5-bb84-05d33e21d3c6-iptables-alerter-script\") pod \"iptables-alerter-b59qf\" (UID: \"eca06d87-f10e-47d5-bb84-05d33e21d3c6\") " pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377870    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-run-systemd\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377875    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-os-release\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377888    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-node-log\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377720    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377904    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-run-ovn\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377921    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-run-netns\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377931    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-run-openvswitch\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377934    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15961897-ec82-41cf-8ce5-b036a30b987c-kubelet-config\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377938    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-cni-netd\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377907    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-run-systemd\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.377969    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-run-ovn\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378007    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0b75180-bc1f-42fa-9792-c2de53a22d56-serviceca\") pod \"node-ca-nb5cd\" (UID: \"e0b75180-bc1f-42fa-9792-c2de53a22d56\") " pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.378687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378058    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-ovnkube-script-lib\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378065    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-system-cni-dir\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378063    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bf939ed-2d88-4258-9694-e2b0a924478c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378092    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01f4e8e6-8355-42ec-9f7a-33c4f0d68c90-tmp-dir\") pod \"node-resolver-6f5mx\" (UID: \"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90\") " pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378104    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15961897-ec82-41cf-8ce5-b036a30b987c-kubelet-config\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378125    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-log-socket\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378147    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-system-cni-dir\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378152    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-cni-bin\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378178    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqmk5\" (UniqueName: \"kubernetes.io/projected/e0b75180-bc1f-42fa-9792-c2de53a22d56-kube-api-access-wqmk5\") pod \"node-ca-nb5cd\" (UID: \"e0b75180-bc1f-42fa-9792-c2de53a22d56\") " pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378232    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-cnibin\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378268    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-cni-bin\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378277    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-slash\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378309    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsq9b\" (UniqueName: \"kubernetes.io/projected/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-kube-api-access-vsq9b\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378314    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-log-socket\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378336    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bf939ed-2d88-4258-9694-e2b0a924478c-cni-binary-copy\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378351    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-slash\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378358    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-cnibin\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.379313 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378363    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378381    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01f4e8e6-8355-42ec-9f7a-33c4f0d68c90-tmp-dir\") pod \"node-resolver-6f5mx\" (UID: \"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90\") " pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378400    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-ovnkube-config\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378307    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-env-overrides\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378400    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsmq7\" (UniqueName: \"kubernetes.io/projected/9d8d82fd-3ead-4428-896e-103c296da485-kube-api-access-wsmq7\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378444    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-kubelet\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378467    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0b75180-bc1f-42fa-9792-c2de53a22d56-host\") pod \"node-ca-nb5cd\" (UID: \"e0b75180-bc1f-42fa-9792-c2de53a22d56\") " pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378490    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378515    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/74387622-269d-4e67-af29-776fabd1e176-konnectivity-ca\") pod \"konnectivity-agent-lxsxf\" (UID: \"74387622-269d-4e67-af29-776fabd1e176\") " pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378556    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-host-kubelet\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.378580    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378596    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0b75180-bc1f-42fa-9792-c2de53a22d56-host\") pod \"node-ca-nb5cd\" (UID: \"e0b75180-bc1f-42fa-9792-c2de53a22d56\") " pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378491    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bf939ed-2d88-4258-9694-e2b0a924478c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.378628    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret podName:15961897-ec82-41cf-8ce5-b036a30b987c nodeName:}" failed. No retries permitted until 2026-07-26 16:08:25.878614007 +0000 UTC m=+2.061698272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret") pod "global-pull-secret-syncer-5dfgr" (UID: "15961897-ec82-41cf-8ce5-b036a30b987c") : object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378951    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/74387622-269d-4e67-af29-776fabd1e176-konnectivity-ca\") pod \"konnectivity-agent-lxsxf\" (UID: \"74387622-269d-4e67-af29-776fabd1e176\") " pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:25.379813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.378951    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bf939ed-2d88-4258-9694-e2b0a924478c-cni-binary-copy\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.380244 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.380224    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-ovn-node-metrics-cert\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.380774 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.380758    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/74387622-269d-4e67-af29-776fabd1e176-agent-certs\") pod \"konnectivity-agent-lxsxf\" (UID: \"74387622-269d-4e67-af29-776fabd1e176\") " pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:25.387100 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.387081    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb7mc\" (UniqueName: \"kubernetes.io/projected/5bf939ed-2d88-4258-9694-e2b0a924478c-kube-api-access-jb7mc\") pod \"multus-additional-cni-plugins-wb6b9\" (UID: \"5bf939ed-2d88-4258-9694-e2b0a924478c\") " pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.387817 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.387793    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsmq7\" (UniqueName: \"kubernetes.io/projected/9d8d82fd-3ead-4428-896e-103c296da485-kube-api-access-wsmq7\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:25.387924 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.387903    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqmk5\" (UniqueName: \"kubernetes.io/projected/e0b75180-bc1f-42fa-9792-c2de53a22d56-kube-api-access-wqmk5\") pod \"node-ca-nb5cd\" (UID: \"e0b75180-bc1f-42fa-9792-c2de53a22d56\") " pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.387981 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.387946    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b5k8\" (UniqueName: \"kubernetes.io/projected/eca06d87-f10e-47d5-bb84-05d33e21d3c6-kube-api-access-4b5k8\") pod \"iptables-alerter-b59qf\" (UID: \"eca06d87-f10e-47d5-bb84-05d33e21d3c6\") " pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.388290 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.388275    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsq9b\" (UniqueName: \"kubernetes.io/projected/6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5-kube-api-access-vsq9b\") pod \"ovnkube-node-dn7k8\" (UID: \"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.388339 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.388315    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77kph\" (UniqueName: \"kubernetes.io/projected/01f4e8e6-8355-42ec-9f7a-33c4f0d68c90-kube-api-access-77kph\") pod \"node-resolver-6f5mx\" (UID: \"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90\") " pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.507453 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.507381    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26"
Jul 26 16:08:25.518058 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.518035    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bafccda_1228_489e_a79b_88d0d1455a82.slice/crio-c09ad58dd9939657977752f444d1b42fa4f69a50f6b3626930f582f0a144c3fa WatchSource:0}: Error finding container c09ad58dd9939657977752f444d1b42fa4f69a50f6b3626930f582f0a144c3fa: Status 404 returned error can't find the container with id c09ad58dd9939657977752f444d1b42fa4f69a50f6b3626930f582f0a144c3fa
Jul 26 16:08:25.524389 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.524370    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5w646"
Jul 26 16:08:25.532078 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.532060    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t79zb"
Jul 26 16:08:25.532742 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.532716    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6954dc32_3e67_4813_9eb7_ea925571876e.slice/crio-c6667417d0b4491f989da849080c43357f4bd9c855cfe5ed7bd4c523ee1cca9f WatchSource:0}: Error finding container c6667417d0b4491f989da849080c43357f4bd9c855cfe5ed7bd4c523ee1cca9f: Status 404 returned error can't find the container with id c6667417d0b4491f989da849080c43357f4bd9c855cfe5ed7bd4c523ee1cca9f
Jul 26 16:08:25.539344 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.539318    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b5a38b_aed2_446f_8f8e_615b1f9557e7.slice/crio-74e5fa01a432d9ac1a2ee56d71ab4e2a62621358ffa622fd6e18de4cefe379b9 WatchSource:0}: Error finding container 74e5fa01a432d9ac1a2ee56d71ab4e2a62621358ffa622fd6e18de4cefe379b9: Status 404 returned error can't find the container with id 74e5fa01a432d9ac1a2ee56d71ab4e2a62621358ffa622fd6e18de4cefe379b9
Jul 26 16:08:25.543335 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.543318    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wb6b9"
Jul 26 16:08:25.552049 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.551988    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf939ed_2d88_4258_9694_e2b0a924478c.slice/crio-b8e137b316b06426db0308348815141d38d53b403e4360cf0f38945e81de41cb WatchSource:0}: Error finding container b8e137b316b06426db0308348815141d38d53b403e4360cf0f38945e81de41cb: Status 404 returned error can't find the container with id b8e137b316b06426db0308348815141d38d53b403e4360cf0f38945e81de41cb
Jul 26 16:08:25.558960 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.558942    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:25.566275 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.566252    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d79cbbc_e4cb_4de2_ba7b_f2043363b9f5.slice/crio-c09c384cd32d908edfb6f159e7f8e6944a3e0d97882824f256272bf03f56f976 WatchSource:0}: Error finding container c09c384cd32d908edfb6f159e7f8e6944a3e0d97882824f256272bf03f56f976: Status 404 returned error can't find the container with id c09c384cd32d908edfb6f159e7f8e6944a3e0d97882824f256272bf03f56f976
Jul 26 16:08:25.572615 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.572596    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nb5cd"
Jul 26 16:08:25.580174 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.580154    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0b75180_bc1f_42fa_9792_c2de53a22d56.slice/crio-b03d4f65ed625b903ed5f355a9c187b4188849d9a496baa997d8b50d4512a721 WatchSource:0}: Error finding container b03d4f65ed625b903ed5f355a9c187b4188849d9a496baa997d8b50d4512a721: Status 404 returned error can't find the container with id b03d4f65ed625b903ed5f355a9c187b4188849d9a496baa997d8b50d4512a721
Jul 26 16:08:25.588043 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.588027    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6f5mx"
Jul 26 16:08:25.595444 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.595423    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01f4e8e6_8355_42ec_9f7a_33c4f0d68c90.slice/crio-dfbec57da5d8d54875fc449f2835fb3c604c1f679b961656bf3aaa60b5a8927f WatchSource:0}: Error finding container dfbec57da5d8d54875fc449f2835fb3c604c1f679b961656bf3aaa60b5a8927f: Status 404 returned error can't find the container with id dfbec57da5d8d54875fc449f2835fb3c604c1f679b961656bf3aaa60b5a8927f
Jul 26 16:08:25.603417 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.603401    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:25.609081 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.609065    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b59qf"
Jul 26 16:08:25.611397 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.611377    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74387622_269d_4e67_af29_776fabd1e176.slice/crio-1af7a1ce0c0c4a3f9f291f31631dd9b3b3ef8f5e691fa16b8b63cb19df9ed24e WatchSource:0}: Error finding container 1af7a1ce0c0c4a3f9f291f31631dd9b3b3ef8f5e691fa16b8b63cb19df9ed24e: Status 404 returned error can't find the container with id 1af7a1ce0c0c4a3f9f291f31631dd9b3b3ef8f5e691fa16b8b63cb19df9ed24e
Jul 26 16:08:25.616795 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:08:25.616778    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca06d87_f10e_47d5_bb84_05d33e21d3c6.slice/crio-33a9ed82680369ab4af205311103c38fadab5389594904f9b4e8fdfb9da20d97 WatchSource:0}: Error finding container 33a9ed82680369ab4af205311103c38fadab5389594904f9b4e8fdfb9da20d97: Status 404 returned error can't find the container with id 33a9ed82680369ab4af205311103c38fadab5389594904f9b4e8fdfb9da20d97
Jul 26 16:08:25.782117 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.782039    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdb5s\" (UniqueName: \"kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s\") pod \"network-check-target-djgt6\" (UID: \"f7f5558e-066e-4005-9d86-3090d19d3d66\") " pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:25.782253 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.782208    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 16:08:25.782253 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.782226    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 16:08:25.782253 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.782235    2570 projected.go:196] Error preparing data for projected volume kube-api-access-tdb5s for pod openshift-network-diagnostics/network-check-target-djgt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:25.782371 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.782285    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s podName:f7f5558e-066e-4005-9d86-3090d19d3d66 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:26.78226754 +0000 UTC m=+2.965351818 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdb5s" (UniqueName: "kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s") pod "network-check-target-djgt6" (UID: "f7f5558e-066e-4005-9d86-3090d19d3d66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:25.883202 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.883160    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:25.883356 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:25.883224    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:25.883433 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.883356    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:25.883433 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.883366    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:25.883433 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.883412    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret podName:15961897-ec82-41cf-8ce5-b036a30b987c nodeName:}" failed. No retries permitted until 2026-07-26 16:08:26.883393914 +0000 UTC m=+3.066478185 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret") pod "global-pull-secret-syncer-5dfgr" (UID: "15961897-ec82-41cf-8ce5-b036a30b987c") : object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:25.883433 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:25.883429    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs podName:9d8d82fd-3ead-4428-896e-103c296da485 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:26.883419431 +0000 UTC m=+3.066503699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs") pod "network-metrics-daemon-7xxbv" (UID: "9d8d82fd-3ead-4428-896e-103c296da485") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:26.211104 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.210999    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 16:03:25 +0000 UTC" deadline="2028-04-04 10:58:02.411033494 +0000 UTC"
Jul 26 16:08:26.211104 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.211053    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14826h49m36.199983722s"
Jul 26 16:08:26.320930 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.320838    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t79zb" event={"ID":"66b5a38b-aed2-446f-8f8e-615b1f9557e7","Type":"ContainerStarted","Data":"74e5fa01a432d9ac1a2ee56d71ab4e2a62621358ffa622fd6e18de4cefe379b9"}
Jul 26 16:08:26.341731 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.341660    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"31b67110ee781749d3f159efc4905833","Type":"ContainerStarted","Data":"6471ba0e2e640cd2981d2408726a2616b2c28eaf77f05382858d1cd55a8d8b35"}
Jul 26 16:08:26.379738 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.379668    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" event={"ID":"a3de1b13181d39474231213cb35e8b2d","Type":"ContainerStarted","Data":"16e7eed5cd07ae5f8e57e5a754eade4518d00480c21a426e0980ed372c5e5398"}
Jul 26 16:08:26.400786 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.400753    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b59qf" event={"ID":"eca06d87-f10e-47d5-bb84-05d33e21d3c6","Type":"ContainerStarted","Data":"33a9ed82680369ab4af205311103c38fadab5389594904f9b4e8fdfb9da20d97"}
Jul 26 16:08:26.418532 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.418500    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lxsxf" event={"ID":"74387622-269d-4e67-af29-776fabd1e176","Type":"ContainerStarted","Data":"1af7a1ce0c0c4a3f9f291f31631dd9b3b3ef8f5e691fa16b8b63cb19df9ed24e"}
Jul 26 16:08:26.449515 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.449427    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6f5mx" event={"ID":"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90","Type":"ContainerStarted","Data":"dfbec57da5d8d54875fc449f2835fb3c604c1f679b961656bf3aaa60b5a8927f"}
Jul 26 16:08:26.477151 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.477059    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nb5cd" event={"ID":"e0b75180-bc1f-42fa-9792-c2de53a22d56","Type":"ContainerStarted","Data":"b03d4f65ed625b903ed5f355a9c187b4188849d9a496baa997d8b50d4512a721"}
Jul 26 16:08:26.487870 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.487838    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" event={"ID":"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5","Type":"ContainerStarted","Data":"c09c384cd32d908edfb6f159e7f8e6944a3e0d97882824f256272bf03f56f976"}
Jul 26 16:08:26.526714 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.526673    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb6b9" event={"ID":"5bf939ed-2d88-4258-9694-e2b0a924478c","Type":"ContainerStarted","Data":"b8e137b316b06426db0308348815141d38d53b403e4360cf0f38945e81de41cb"}
Jul 26 16:08:26.544422 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.544341    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5w646" event={"ID":"6954dc32-3e67-4813-9eb7-ea925571876e","Type":"ContainerStarted","Data":"c6667417d0b4491f989da849080c43357f4bd9c855cfe5ed7bd4c523ee1cca9f"}
Jul 26 16:08:26.561516 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.561373    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26" event={"ID":"8bafccda-1228-489e-a79b-88d0d1455a82","Type":"ContainerStarted","Data":"c09ad58dd9939657977752f444d1b42fa4f69a50f6b3626930f582f0a144c3fa"}
Jul 26 16:08:26.599477 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.599284    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 16:08:26.789979 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.789818    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdb5s\" (UniqueName: \"kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s\") pod \"network-check-target-djgt6\" (UID: \"f7f5558e-066e-4005-9d86-3090d19d3d66\") " pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:26.790134 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:26.790024    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 16:08:26.790134 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:26.790044    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 16:08:26.790134 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:26.790057    2570 projected.go:196] Error preparing data for projected volume kube-api-access-tdb5s for pod openshift-network-diagnostics/network-check-target-djgt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:26.790134 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:26.790118    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s podName:f7f5558e-066e-4005-9d86-3090d19d3d66 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:28.790098809 +0000 UTC m=+4.973183075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdb5s" (UniqueName: "kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s") pod "network-check-target-djgt6" (UID: "f7f5558e-066e-4005-9d86-3090d19d3d66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:26.890879 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.890841    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:26.891045 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:26.890900    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:26.891045 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:26.891038    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:26.891795 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:26.891097    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs podName:9d8d82fd-3ead-4428-896e-103c296da485 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:28.891077883 +0000 UTC m=+5.074162172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs") pod "network-metrics-daemon-7xxbv" (UID: "9d8d82fd-3ead-4428-896e-103c296da485") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:26.891795 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:26.891475    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:26.891795 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:26.891527    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret podName:15961897-ec82-41cf-8ce5-b036a30b987c nodeName:}" failed. No retries permitted until 2026-07-26 16:08:28.891511965 +0000 UTC m=+5.074596234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret") pod "global-pull-secret-syncer-5dfgr" (UID: "15961897-ec82-41cf-8ce5-b036a30b987c") : object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:27.213766 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:27.211924    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 16:03:25 +0000 UTC" deadline="2028-03-08 07:50:09.201006105 +0000 UTC"
Jul 26 16:08:27.213766 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:27.211979    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14175h41m41.989030617s"
Jul 26 16:08:27.294500 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:27.294472    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:27.294500 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:27.294496    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:27.294743 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:27.294607    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:27.294743 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:27.294660    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:27.294873 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:27.294799    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:27.294929 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:27.294899    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:27.623445 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:27.623367    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 16:08:28.807542 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:28.807500    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdb5s\" (UniqueName: \"kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s\") pod \"network-check-target-djgt6\" (UID: \"f7f5558e-066e-4005-9d86-3090d19d3d66\") " pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:28.807985 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:28.807696    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 16:08:28.807985 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:28.807715    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 16:08:28.807985 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:28.807728    2570 projected.go:196] Error preparing data for projected volume kube-api-access-tdb5s for pod openshift-network-diagnostics/network-check-target-djgt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:28.807985 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:28.807784    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s podName:f7f5558e-066e-4005-9d86-3090d19d3d66 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:32.807766315 +0000 UTC m=+8.990850587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdb5s" (UniqueName: "kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s") pod "network-check-target-djgt6" (UID: "f7f5558e-066e-4005-9d86-3090d19d3d66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:28.908432 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:28.908398    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:28.908600 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:28.908457    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:28.908665 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:28.908620    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:28.908724 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:28.908679    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs podName:9d8d82fd-3ead-4428-896e-103c296da485 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:32.908658003 +0000 UTC m=+9.091742295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs") pod "network-metrics-daemon-7xxbv" (UID: "9d8d82fd-3ead-4428-896e-103c296da485") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:28.909063 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:28.909043    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:28.909154 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:28.909097    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret podName:15961897-ec82-41cf-8ce5-b036a30b987c nodeName:}" failed. No retries permitted until 2026-07-26 16:08:32.909081347 +0000 UTC m=+9.092165616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret") pod "global-pull-secret-syncer-5dfgr" (UID: "15961897-ec82-41cf-8ce5-b036a30b987c") : object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:29.294167 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:29.294097    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:29.294380 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:29.294245    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:29.294558 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:29.294532    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:29.294671 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:29.294649    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:29.294797 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:29.294536    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:29.294797 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:29.294755    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:31.294529 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:31.294441    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:31.294529 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:31.294487    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:31.294999 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:31.294590    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:31.294999 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:31.294441    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:31.295067 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:31.295044    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:31.295173 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:31.295136    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:32.841567 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:32.841477    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdb5s\" (UniqueName: \"kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s\") pod \"network-check-target-djgt6\" (UID: \"f7f5558e-066e-4005-9d86-3090d19d3d66\") " pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:32.842031 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:32.841686    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 16:08:32.842031 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:32.841701    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 16:08:32.842031 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:32.841713    2570 projected.go:196] Error preparing data for projected volume kube-api-access-tdb5s for pod openshift-network-diagnostics/network-check-target-djgt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:32.842031 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:32.841774    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s podName:f7f5558e-066e-4005-9d86-3090d19d3d66 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:40.841757268 +0000 UTC m=+17.024841537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdb5s" (UniqueName: "kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s") pod "network-check-target-djgt6" (UID: "f7f5558e-066e-4005-9d86-3090d19d3d66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:32.942367 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:32.942092    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:32.942367 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:32.942153    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:32.942367 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:32.942282    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:32.942367 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:32.942310    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:32.942367 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:32.942363    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret podName:15961897-ec82-41cf-8ce5-b036a30b987c nodeName:}" failed. No retries permitted until 2026-07-26 16:08:40.942340322 +0000 UTC m=+17.125424590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret") pod "global-pull-secret-syncer-5dfgr" (UID: "15961897-ec82-41cf-8ce5-b036a30b987c") : object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:32.942772 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:32.942386    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs podName:9d8d82fd-3ead-4428-896e-103c296da485 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:40.942374977 +0000 UTC m=+17.125459243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs") pod "network-metrics-daemon-7xxbv" (UID: "9d8d82fd-3ead-4428-896e-103c296da485") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:33.294267 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:33.294226    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:33.294267 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:33.294248    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:33.294512 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:33.294348    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:33.294718 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:33.294692    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:33.294839 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:33.294751    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:33.294908 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:33.294842    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:35.294588 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:35.294475    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:35.294588 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:35.294521    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:35.295102 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:35.294615    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:35.295102 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:35.294707    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:35.295102 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:35.294778    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:35.295102 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:35.294864    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:37.295028 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:37.294981    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:37.295463 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:37.294981    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:37.295463 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:37.295127    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:37.295463 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:37.295152    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:37.295463 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:37.294981    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:37.295463 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:37.295282    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:39.294827 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:39.294782    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:39.294827 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:39.294807    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:39.295330 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:39.294783    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:39.295330 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:39.294919    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:39.295330 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:39.294989    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:39.295330 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:39.295055    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:40.903353 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:40.903326    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdb5s\" (UniqueName: \"kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s\") pod \"network-check-target-djgt6\" (UID: \"f7f5558e-066e-4005-9d86-3090d19d3d66\") " pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:40.903624 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:40.903475    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 16:08:40.903624 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:40.903491    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 16:08:40.903624 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:40.903503    2570 projected.go:196] Error preparing data for projected volume kube-api-access-tdb5s for pod openshift-network-diagnostics/network-check-target-djgt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:40.903624 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:40.903549    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s podName:f7f5558e-066e-4005-9d86-3090d19d3d66 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:56.903536144 +0000 UTC m=+33.086620409 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdb5s" (UniqueName: "kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s") pod "network-check-target-djgt6" (UID: "f7f5558e-066e-4005-9d86-3090d19d3d66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:41.003897 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:41.003860    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:41.004049 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:41.003921    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:41.004049 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:41.004034    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:41.004139 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:41.004102    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret podName:15961897-ec82-41cf-8ce5-b036a30b987c nodeName:}" failed. No retries permitted until 2026-07-26 16:08:57.004087433 +0000 UTC m=+33.187171698 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret") pod "global-pull-secret-syncer-5dfgr" (UID: "15961897-ec82-41cf-8ce5-b036a30b987c") : object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:41.004139 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:41.004041    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:41.004233 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:41.004175    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs podName:9d8d82fd-3ead-4428-896e-103c296da485 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:57.004161904 +0000 UTC m=+33.187246183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs") pod "network-metrics-daemon-7xxbv" (UID: "9d8d82fd-3ead-4428-896e-103c296da485") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:41.294290 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:41.294256    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:41.294290 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:41.294284    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:41.294501 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:41.294284    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:41.294501 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:41.294378    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:41.294501 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:41.294462    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:41.294641 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:41.294533    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:43.294357 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:43.294325    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:43.294357 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:43.294348    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:43.294941 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:43.294432    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:43.294941 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:43.294455    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:43.294941 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:43.294531    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:43.294941 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:43.294597    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:45.295069 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:45.295045    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:45.295367 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:45.295077    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:45.295367 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:45.295081    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:45.295367 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:45.295146    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:45.295367 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:45.295225    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:45.295367 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:45.295277    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:45.602705 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:45.602619    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5w646" event={"ID":"6954dc32-3e67-4813-9eb7-ea925571876e","Type":"ContainerStarted","Data":"7f88eff90f2894cee2cd49de936cc2557507c431ff05ce93a5a9516fdb93028d"}
Jul 26 16:08:45.604050 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:45.604024    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t79zb" event={"ID":"66b5a38b-aed2-446f-8f8e-615b1f9557e7","Type":"ContainerStarted","Data":"b727011662599bee3359bd2fc17989d84dca5a02a369b1ebeb87d16fee826fdc"}
Jul 26 16:08:45.605322 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:45.605294    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" event={"ID":"a3de1b13181d39474231213cb35e8b2d","Type":"ContainerStarted","Data":"304e758f49467450ba07090665350463040d4587d0334af15ccab4ad9ca42c44"}
Jul 26 16:08:45.620478 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:45.620437    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5w646" podStartSLOduration=1.8667480319999998 podStartE2EDuration="21.620422703s" totalImagesPullingTime="19.753674671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:45.620072686 +0000 UTC m=+21.803156974" watchObservedRunningTime="2026-07-26 16:08:45.620422703 +0000 UTC m=+21.803506991"
Jul 26 16:08:45.651125 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:45.651062    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" podStartSLOduration=20.651043088 podStartE2EDuration="20.651043088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:45.633616416 +0000 UTC m=+21.816700703" watchObservedRunningTime="2026-07-26 16:08:45.651043088 +0000 UTC m=+21.834127376"
Jul 26 16:08:45.651530 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:45.651491    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t79zb" podStartSLOduration=1.929918985 podStartE2EDuration="21.651479432s" totalImagesPullingTime="19.721560447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:45.650941245 +0000 UTC m=+21.834025532" watchObservedRunningTime="2026-07-26 16:08:45.651479432 +0000 UTC m=+21.834563721"
Jul 26 16:08:46.609251 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.609000    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lxsxf" event={"ID":"74387622-269d-4e67-af29-776fabd1e176","Type":"ContainerStarted","Data":"4f9675c54cb405cb151f0f5c71999b5d70a5490362bc6790923482c93805f64a"}
Jul 26 16:08:46.610618 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.610596    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6f5mx" event={"ID":"01f4e8e6-8355-42ec-9f7a-33c4f0d68c90","Type":"ContainerStarted","Data":"2a7c123f0f70bea52767857d6036d5f0472c7e631df5d47d8dd2d1130a828f70"}
Jul 26 16:08:46.612041 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.612003    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nb5cd" event={"ID":"e0b75180-bc1f-42fa-9792-c2de53a22d56","Type":"ContainerStarted","Data":"b1273334cf49f3e7017353332220fd8181fcb4699aa0c4fff8a0ae4598650a82"}
Jul 26 16:08:46.614938 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.614921    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:08:46.615330 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.615308    2570 generic.go:356] "Generic (PLEG): container finished" podID="6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5" containerID="2e230217df674b364e1ca448cc92955f21acf1255f93dc02def195be1653ad3e" exitCode=1
Jul 26 16:08:46.615404 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.615369    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" event={"ID":"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5","Type":"ContainerStarted","Data":"bfb73ca2146dc158ab08deabcfc2aba6b055d80c4fa4ff338c9dd69d2978d257"}
Jul 26 16:08:46.615457 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.615433    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" event={"ID":"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5","Type":"ContainerStarted","Data":"a957cb8407aed6d97ce34c4fb67cc74e0348d0824239c64499b86c0a781f4fb8"}
Jul 26 16:08:46.615508 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.615484    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" event={"ID":"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5","Type":"ContainerStarted","Data":"bae2f27e9cc60ce0c9e372eb721c25ebe0e200f3004a587ecfaebb4398b6cac5"}
Jul 26 16:08:46.615554 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.615534    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" event={"ID":"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5","Type":"ContainerStarted","Data":"9a284970fd477edc35f076ab204c23d57490995cb26c0fc76f8e3afe385d543f"}
Jul 26 16:08:46.615589 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.615579    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" event={"ID":"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5","Type":"ContainerDied","Data":"2e230217df674b364e1ca448cc92955f21acf1255f93dc02def195be1653ad3e"}
Jul 26 16:08:46.615634 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.615616    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" event={"ID":"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5","Type":"ContainerStarted","Data":"5cc62dff05c3b85bde833ca0caab901705831fc140e5ec76cf1840bb879ab24e"}
Jul 26 16:08:46.616999 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.616977    2570 generic.go:356] "Generic (PLEG): container finished" podID="5bf939ed-2d88-4258-9694-e2b0a924478c" containerID="0a41f256f0cd46a0edce69308c9365f68e68b4b0296d71354d78fa7afb472738" exitCode=0
Jul 26 16:08:46.617072 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.617051    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb6b9" event={"ID":"5bf939ed-2d88-4258-9694-e2b0a924478c","Type":"ContainerDied","Data":"0a41f256f0cd46a0edce69308c9365f68e68b4b0296d71354d78fa7afb472738"}
Jul 26 16:08:46.618606 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.618586    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26" event={"ID":"8bafccda-1228-489e-a79b-88d0d1455a82","Type":"ContainerStarted","Data":"f9a6803c828e8d41b080b68ccd32f2d614d26b3970e388afa062de08a0d69b83"}
Jul 26 16:08:46.620257 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.620234    2570 generic.go:356] "Generic (PLEG): container finished" podID="31b67110ee781749d3f159efc4905833" containerID="3bd8633bbb72e0d9bad2156e03a1f4cb438e9a2a84ee3481b12b30fff2315482" exitCode=0
Jul 26 16:08:46.620353 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.620286    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"31b67110ee781749d3f159efc4905833","Type":"ContainerDied","Data":"3bd8633bbb72e0d9bad2156e03a1f4cb438e9a2a84ee3481b12b30fff2315482"}
Jul 26 16:08:46.628808 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.628773    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lxsxf" podStartSLOduration=3.042273638 podStartE2EDuration="22.628761494s" totalImagesPullingTime="19.586487856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:46.628656083 +0000 UTC m=+22.811740400" watchObservedRunningTime="2026-07-26 16:08:46.628761494 +0000 UTC m=+22.811845780"
Jul 26 16:08:46.644365 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.644339    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:46.663520 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.663471    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nb5cd" podStartSLOduration=3.015469618 podStartE2EDuration="22.663451547s" totalImagesPullingTime="19.647981929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:46.662731185 +0000 UTC m=+22.845815487" watchObservedRunningTime="2026-07-26 16:08:46.663451547 +0000 UTC m=+22.846535835"
Jul 26 16:08:46.691333 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:46.691291    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6f5mx" podStartSLOduration=3.088883646 podStartE2EDuration="22.691276909s" totalImagesPullingTime="19.602393263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:46.67733229 +0000 UTC m=+22.860416579" watchObservedRunningTime="2026-07-26 16:08:46.691276909 +0000 UTC m=+22.874361196"
Jul 26 16:08:47.294486 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:47.294449    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:47.294708 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:47.294457    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:47.294708 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:47.294457    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:47.294708 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:47.294646    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:47.294896 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:47.294621    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:47.294896 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:47.294744    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:47.624228 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:47.624181    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"31b67110ee781749d3f159efc4905833","Type":"ContainerStarted","Data":"1a4bc58494c8b6ef9e8fee2fd191af9367483dcd10b3a829fdaf808df7bbd578"}
Jul 26 16:08:47.626649 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:47.626293    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b59qf" event={"ID":"eca06d87-f10e-47d5-bb84-05d33e21d3c6","Type":"ContainerStarted","Data":"c19eab6fbdd660fc5b9ab1719259138f730beefa1642c41655300ecab6b951b2"}
Jul 26 16:08:47.641805 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:47.641767    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" podStartSLOduration=22.641751289 podStartE2EDuration="22.641751289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:47.6415305 +0000 UTC m=+23.824614837" watchObservedRunningTime="2026-07-26 16:08:47.641751289 +0000 UTC m=+23.824835578"
Jul 26 16:08:47.657062 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:47.657017    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b59qf" podStartSLOduration=4.015096681 podStartE2EDuration="23.656998968s" totalImagesPullingTime="19.641902287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:47.656930484 +0000 UTC m=+23.840014784" watchObservedRunningTime="2026-07-26 16:08:47.656998968 +0000 UTC m=+23.840083256"
Jul 26 16:08:47.696752 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:47.696720    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 26 16:08:48.248051 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:48.247916    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-26T16:08:47.69673715Z","UUID":"c4a89b87-9aea-43b7-a586-80e655f08b7a","Handler":null,"Name":"","Endpoint":""}
Jul 26 16:08:48.249854 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:48.249828    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 26 16:08:48.249982 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:48.249877    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 26 16:08:48.632089 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:48.631301    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26" event={"ID":"8bafccda-1228-489e-a79b-88d0d1455a82","Type":"ContainerStarted","Data":"c49fd4005c3a22c69f2d8462eed65cb254801470b52448d07295b05b35a7bc67"}
Jul 26 16:08:49.294232 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:49.294198    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:49.294472 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:49.294199    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:49.294472 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:49.294333    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:49.294472 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:49.294404    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:49.294472 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:49.294206    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:49.294660 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:49.294495    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:49.636598 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:49.636533    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:08:49.636994 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:49.636940    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" event={"ID":"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5","Type":"ContainerStarted","Data":"d690e3e3402bddd5e40967dac2928542f06cf433c3542a18ba5138b375889148"}
Jul 26 16:08:49.639180 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:49.639121    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26" event={"ID":"8bafccda-1228-489e-a79b-88d0d1455a82","Type":"ContainerStarted","Data":"092b0937b1ac6cf6df6eda4ac0966cb4fb2a72cb613a4a5328bcffd3e379a225"}
Jul 26 16:08:49.660517 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:49.660466    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzz26" podStartSLOduration=3.675480765 podStartE2EDuration="25.660448315s" totalImagesPullingTime="21.98496755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:49.659895146 +0000 UTC m=+25.842979453" watchObservedRunningTime="2026-07-26 16:08:49.660448315 +0000 UTC m=+25.843532603"
Jul 26 16:08:50.201633 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:50.201593    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:50.202348 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:50.202330    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:50.641894 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:50.641829    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lxsxf"
Jul 26 16:08:51.294906 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.294877    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:51.295069 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.294971    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:51.295069 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:51.294977    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:51.295069 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.295001    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:51.295171 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:51.295076    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:51.295171 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:51.295134    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:51.646739 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.646659    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:08:51.647603 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.647004    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" event={"ID":"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5","Type":"ContainerStarted","Data":"f696747b3bc551094bedf4b993d5e7051503c246dfcab8d64361d1dd53923047"}
Jul 26 16:08:51.647603 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.647251    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:51.647603 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.647309    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:51.647603 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.647537    2570 scope.go:117] "RemoveContainer" containerID="2e230217df674b364e1ca448cc92955f21acf1255f93dc02def195be1653ad3e"
Jul 26 16:08:51.648852 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.648831    2570 generic.go:356] "Generic (PLEG): container finished" podID="5bf939ed-2d88-4258-9694-e2b0a924478c" containerID="2dbec95035f52e7dad55a95f91554271170933b5ce2e49cb3968543c1c6045fd" exitCode=0
Jul 26 16:08:51.648971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.648913    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb6b9" event={"ID":"5bf939ed-2d88-4258-9694-e2b0a924478c","Type":"ContainerDied","Data":"2dbec95035f52e7dad55a95f91554271170933b5ce2e49cb3968543c1c6045fd"}
Jul 26 16:08:51.662641 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:51.662623    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:52.654846 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.654686    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:08:52.655237 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.655171    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" event={"ID":"6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5","Type":"ContainerStarted","Data":"8c97c681b177fa7c99b65136f6ba41410a04aba277b1e3809e4460beb52fcaef"}
Jul 26 16:08:52.655370 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.655353    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:52.671073 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.671047    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:08:52.685824 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.685779    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8" podStartSLOduration=8.401042193 podStartE2EDuration="28.685765857s" totalImagesPullingTime="20.284723664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:52.683821013 +0000 UTC m=+28.866905365" watchObservedRunningTime="2026-07-26 16:08:52.685765857 +0000 UTC m=+28.868850144"
Jul 26 16:08:52.818906 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.818876    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7xxbv"]
Jul 26 16:08:52.819058 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.819043    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:52.819203 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:52.819167    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:52.825149 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.825046    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5dfgr"]
Jul 26 16:08:52.825286 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.825236    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:52.825457 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:52.825402    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:52.826954 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.826544    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-djgt6"]
Jul 26 16:08:52.826954 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:52.826631    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:52.826954 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:52.826713    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:53.105732 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:53.105659    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6f5mx_01f4e8e6-8355-42ec-9f7a-33c4f0d68c90/dns-node-resolver/0.log"
Jul 26 16:08:53.659244 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:53.659206    2570 generic.go:356] "Generic (PLEG): container finished" podID="5bf939ed-2d88-4258-9694-e2b0a924478c" containerID="5a7df1277a1f0c698b29764c2cc4593b2f15f9991bef05849d3995a1121dc007" exitCode=0
Jul 26 16:08:53.659623 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:53.659245    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb6b9" event={"ID":"5bf939ed-2d88-4258-9694-e2b0a924478c","Type":"ContainerDied","Data":"5a7df1277a1f0c698b29764c2cc4593b2f15f9991bef05849d3995a1121dc007"}
Jul 26 16:08:54.281202 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:54.281168    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nb5cd_e0b75180-bc1f-42fa-9792-c2de53a22d56/node-ca/0.log"
Jul 26 16:08:54.297266 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:54.297241    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:54.297266 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:54.297259    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:54.297487 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:54.297241    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:54.297487 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:54.297340    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:54.297487 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:54.297403    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:54.297487 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:54.297481    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:55.666523 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:55.666269    2570 generic.go:356] "Generic (PLEG): container finished" podID="5bf939ed-2d88-4258-9694-e2b0a924478c" containerID="f95d6c0dd258f658cdcecf51fdbfd787d8d2264921de87c2f4d293c7611f7ae4" exitCode=0
Jul 26 16:08:55.666911 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:55.666349    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb6b9" event={"ID":"5bf939ed-2d88-4258-9694-e2b0a924478c","Type":"ContainerDied","Data":"f95d6c0dd258f658cdcecf51fdbfd787d8d2264921de87c2f4d293c7611f7ae4"}
Jul 26 16:08:56.298012 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:56.297982    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:56.298177 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:56.298105    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:56.298250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:56.298112    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:56.298250 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:56.298224    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:56.298318 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:56.298108    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:56.298353 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:56.298312    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:08:56.922138 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:56.922103    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdb5s\" (UniqueName: \"kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s\") pod \"network-check-target-djgt6\" (UID: \"f7f5558e-066e-4005-9d86-3090d19d3d66\") " pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:56.922535 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:56.922278    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 16:08:56.922535 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:56.922295    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 16:08:56.922535 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:56.922304    2570 projected.go:196] Error preparing data for projected volume kube-api-access-tdb5s for pod openshift-network-diagnostics/network-check-target-djgt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:56.922535 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:56.922359    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s podName:f7f5558e-066e-4005-9d86-3090d19d3d66 nodeName:}" failed. No retries permitted until 2026-07-26 16:09:28.922341843 +0000 UTC m=+65.105426108 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdb5s" (UniqueName: "kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s") pod "network-check-target-djgt6" (UID: "f7f5558e-066e-4005-9d86-3090d19d3d66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 16:08:57.022927 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:57.022887    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:57.023085 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:57.022944    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:57.023085 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:57.023039    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:57.023085 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:57.023054    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:57.023250 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:57.023112    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret podName:15961897-ec82-41cf-8ce5-b036a30b987c nodeName:}" failed. No retries permitted until 2026-07-26 16:09:29.023095403 +0000 UTC m=+65.206179682 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret") pod "global-pull-secret-syncer-5dfgr" (UID: "15961897-ec82-41cf-8ce5-b036a30b987c") : object "kube-system"/"original-pull-secret" not registered
Jul 26 16:08:57.023250 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:57.023129    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs podName:9d8d82fd-3ead-4428-896e-103c296da485 nodeName:}" failed. No retries permitted until 2026-07-26 16:09:29.023120913 +0000 UTC m=+65.206205178 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs") pod "network-metrics-daemon-7xxbv" (UID: "9d8d82fd-3ead-4428-896e-103c296da485") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 16:08:58.297648 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:58.297607    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:08:58.298073 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:58.297675    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:08:58.298073 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:08:58.297683    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:08:58.298073 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:58.297770    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:08:58.298073 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:58.297817    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:08:58.298073 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:08:58.297934    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:09:00.294472 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:00.294434    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:00.294472 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:00.294470    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:00.295150 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:00.294439    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:00.295150 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:00.294559    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:09:00.295150 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:00.294620    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:09:00.295150 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:00.294674    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:09:00.679147 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:00.679058    2570 generic.go:356] "Generic (PLEG): container finished" podID="5bf939ed-2d88-4258-9694-e2b0a924478c" containerID="afcee99e9fad39e27a1a5c0ef6b6ba718aab1bee0b97e61fc9bc0260a9233dea" exitCode=0
Jul 26 16:09:00.679147 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:00.679108    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb6b9" event={"ID":"5bf939ed-2d88-4258-9694-e2b0a924478c","Type":"ContainerDied","Data":"afcee99e9fad39e27a1a5c0ef6b6ba718aab1bee0b97e61fc9bc0260a9233dea"}
Jul 26 16:09:01.684838 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:01.684805    2570 generic.go:356] "Generic (PLEG): container finished" podID="5bf939ed-2d88-4258-9694-e2b0a924478c" containerID="9444fc8a89b4769692827d6959c5602ec0145b9fff2416596cc64a8821e8bb50" exitCode=0
Jul 26 16:09:01.684838 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:01.684842    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb6b9" event={"ID":"5bf939ed-2d88-4258-9694-e2b0a924478c","Type":"ContainerDied","Data":"9444fc8a89b4769692827d6959c5602ec0145b9fff2416596cc64a8821e8bb50"}
Jul 26 16:09:02.297263 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:02.297235    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:02.297430 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:02.297325    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:02.297430 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:02.297331    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:09:02.297430 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:02.297350    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:02.297534 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:02.297428    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:09:02.297534 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:02.297516    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:09:02.690808 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:02.690717    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb6b9" event={"ID":"5bf939ed-2d88-4258-9694-e2b0a924478c","Type":"ContainerStarted","Data":"fc599b89a659ff0216437869a0beaeac3b5740affc9ec8d7936ddbb06e4ddee4"}
Jul 26 16:09:02.715518 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:02.715471    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wb6b9" podStartSLOduration=8.693728283 podStartE2EDuration="38.715456313s" totalImagesPullingTime="30.02172803s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:02.715023092 +0000 UTC m=+38.898107381" watchObservedRunningTime="2026-07-26 16:09:02.715456313 +0000 UTC m=+38.898540597"
Jul 26 16:09:04.295570 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:04.295535    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:04.295968 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:04.295631    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:04.295968 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:04.295661    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:04.295968 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:04.295675    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:09:04.295968 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:04.295738    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:09:04.295968 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:04.295818    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:09:06.294246 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:06.294211    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:06.294620 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:06.294213    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:06.294620 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:06.294320    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:09:06.294620 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:06.294211    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:06.294620 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:06.294391    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:09:06.294620 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:06.294513    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:09:08.294242 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:08.294210    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:08.294699 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:08.294211    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:08.294699 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:08.294343    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:09:08.294699 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:08.294213    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:08.294699 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:08.294450    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:09:08.294699 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:08.294386    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:09:10.294118 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:10.294083    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:10.294118 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:10.294113    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:10.294118 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:10.294081    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:10.294636 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:10.294230    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:09:10.294636 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:10.294272    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:09:10.294636 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:10.294319    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:09:12.294551 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:12.294517    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:12.294551 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:12.294549    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:12.295046 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:12.294519    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:12.295046 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:12.294637    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:09:12.295046 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:12.294725    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:09:12.295046 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:12.294836    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:09:14.295011 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:14.294979    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:14.295467 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:14.295059    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:14.295467 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:14.295065    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djgt6" podUID="f7f5558e-066e-4005-9d86-3090d19d3d66"
Jul 26 16:09:14.295467 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:14.295212    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:14.295467 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:14.295210    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xxbv" podUID="9d8d82fd-3ead-4428-896e-103c296da485"
Jul 26 16:09:14.295467 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:14.295297    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dfgr" podUID="15961897-ec82-41cf-8ce5-b036a30b987c"
Jul 26 16:09:15.629827 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.629798    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeReady"
Jul 26 16:09:15.630215 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.629922    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 26 16:09:15.665027 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.664995    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b59467984-gbq79"]
Jul 26 16:09:15.677422 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.677396    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.678912 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.678883    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g48ff"]
Jul 26 16:09:15.680220 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.680183    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 26 16:09:15.680325 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.680242    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 26 16:09:15.680325 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.680261    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 26 16:09:15.680325 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.680306    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-krzsv\""
Jul 26 16:09:15.689982 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.689958    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 26 16:09:15.692713 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.692685    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t9gsq"]
Jul 26 16:09:15.692826 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.692811    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.695654 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.695632    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 26 16:09:15.695654 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.695644    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r6bm5\""
Jul 26 16:09:15.695817 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.695702    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 26 16:09:15.695817 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.695717    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 26 16:09:15.695817 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.695776    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 26 16:09:15.707250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.707087    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b59467984-gbq79"]
Jul 26 16:09:15.707250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.707128    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t9gsq"]
Jul 26 16:09:15.707250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.707162    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g48ff"]
Jul 26 16:09:15.707471 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.707302    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t9gsq"
Jul 26 16:09:15.710161 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.710142    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vf2c6\""
Jul 26 16:09:15.710161 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.710157    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 26 16:09:15.710328 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.710150    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 26 16:09:15.710328 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.710202    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 26 16:09:15.773592 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773555    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-trusted-ca\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.773592 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773596    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85d1cceb-8043-4994-9bfe-8c690d05319e-cert\") pod \"ingress-canary-t9gsq\" (UID: \"85d1cceb-8043-4994-9bfe-8c690d05319e\") " pod="openshift-ingress-canary/ingress-canary-t9gsq"
Jul 26 16:09:15.773816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773624    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b089bff1-c417-441f-9921-521378315b57-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.773816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773667    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnw92\" (UniqueName: \"kubernetes.io/projected/85d1cceb-8043-4994-9bfe-8c690d05319e-kube-api-access-gnw92\") pod \"ingress-canary-t9gsq\" (UID: \"85d1cceb-8043-4994-9bfe-8c690d05319e\") " pod="openshift-ingress-canary/ingress-canary-t9gsq"
Jul 26 16:09:15.773816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773699    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b089bff1-c417-441f-9921-521378315b57-crio-socket\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.773816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773719    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b089bff1-c417-441f-9921-521378315b57-data-volume\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.773816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773757    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-bound-sa-token\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.773816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773780    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjkr8\" (UniqueName: \"kubernetes.io/projected/b089bff1-c417-441f-9921-521378315b57-kube-api-access-cjkr8\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.773816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773800    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-image-registry-private-configuration\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.773816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773819    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b089bff1-c417-441f-9921-521378315b57-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.774094 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773853    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-registry-certificates\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.774094 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773871    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-registry-tls\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.774094 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773902    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-ca-trust-extracted\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.774094 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773934    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smc7f\" (UniqueName: \"kubernetes.io/projected/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-kube-api-access-smc7f\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.774094 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.773964    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-installation-pull-secrets\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.777626 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.777602    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tnrlp"]
Jul 26 16:09:15.803392 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.803362    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tnrlp"]
Jul 26 16:09:15.803547 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.803494    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.806101 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.806080    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 26 16:09:15.806220 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.806209    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 26 16:09:15.806286 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.806273    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-grxdp\""
Jul 26 16:09:15.874443 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874371    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85d1cceb-8043-4994-9bfe-8c690d05319e-cert\") pod \"ingress-canary-t9gsq\" (UID: \"85d1cceb-8043-4994-9bfe-8c690d05319e\") " pod="openshift-ingress-canary/ingress-canary-t9gsq"
Jul 26 16:09:15.874443 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874403    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b089bff1-c417-441f-9921-521378315b57-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.874443 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874425    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnw92\" (UniqueName: \"kubernetes.io/projected/85d1cceb-8043-4994-9bfe-8c690d05319e-kube-api-access-gnw92\") pod \"ingress-canary-t9gsq\" (UID: \"85d1cceb-8043-4994-9bfe-8c690d05319e\") " pod="openshift-ingress-canary/ingress-canary-t9gsq"
Jul 26 16:09:15.874640 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874552    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b089bff1-c417-441f-9921-521378315b57-crio-socket\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.874640 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874583    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b089bff1-c417-441f-9921-521378315b57-data-volume\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.874640 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874610    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411592b1-9f45-453b-a163-f9fdd81b9ef3-metrics-tls\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.874771 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874638    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-bound-sa-token\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.874771 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874667    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjkr8\" (UniqueName: \"kubernetes.io/projected/b089bff1-c417-441f-9921-521378315b57-kube-api-access-cjkr8\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.874771 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874693    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/411592b1-9f45-453b-a163-f9fdd81b9ef3-config-volume\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.874771 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874725    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-image-registry-private-configuration\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.874771 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874750    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b089bff1-c417-441f-9921-521378315b57-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.875021 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874772    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411592b1-9f45-453b-a163-f9fdd81b9ef3-tmp-dir\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.875021 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874807    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b089bff1-c417-441f-9921-521378315b57-crio-socket\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.875021 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874836    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-registry-certificates\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.875021 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874871    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b089bff1-c417-441f-9921-521378315b57-data-volume\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.875021 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.874974    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-registry-tls\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.875021 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.875014    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zs7q\" (UniqueName: \"kubernetes.io/projected/411592b1-9f45-453b-a163-f9fdd81b9ef3-kube-api-access-4zs7q\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.875337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.875067    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-ca-trust-extracted\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.875337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.875094    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smc7f\" (UniqueName: \"kubernetes.io/projected/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-kube-api-access-smc7f\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.875337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.875126    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-installation-pull-secrets\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.875337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.875240    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-trusted-ca\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.875337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.875314    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b089bff1-c417-441f-9921-521378315b57-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.875619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.875584    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-ca-trust-extracted\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.875729 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.875707    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-registry-certificates\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.876420 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.876397    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-trusted-ca\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.878832 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.878810    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b089bff1-c417-441f-9921-521378315b57-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.878905 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.878862    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-installation-pull-secrets\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.878905 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.878875    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85d1cceb-8043-4994-9bfe-8c690d05319e-cert\") pod \"ingress-canary-t9gsq\" (UID: \"85d1cceb-8043-4994-9bfe-8c690d05319e\") " pod="openshift-ingress-canary/ingress-canary-t9gsq"
Jul 26 16:09:15.878905 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.878882    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-registry-tls\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.879007 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.878924    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-image-registry-private-configuration\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.882748 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.882713    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-bound-sa-token\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.882835 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.882745    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjkr8\" (UniqueName: \"kubernetes.io/projected/b089bff1-c417-441f-9921-521378315b57-kube-api-access-cjkr8\") pod \"insights-runtime-extractor-g48ff\" (UID: \"b089bff1-c417-441f-9921-521378315b57\") " pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:15.883381 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.883362    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnw92\" (UniqueName: \"kubernetes.io/projected/85d1cceb-8043-4994-9bfe-8c690d05319e-kube-api-access-gnw92\") pod \"ingress-canary-t9gsq\" (UID: \"85d1cceb-8043-4994-9bfe-8c690d05319e\") " pod="openshift-ingress-canary/ingress-canary-t9gsq"
Jul 26 16:09:15.883447 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.883366    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smc7f\" (UniqueName: \"kubernetes.io/projected/4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8-kube-api-access-smc7f\") pod \"image-registry-7b59467984-gbq79\" (UID: \"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8\") " pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:15.976505 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.976464    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411592b1-9f45-453b-a163-f9fdd81b9ef3-metrics-tls\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.976505 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.976512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/411592b1-9f45-453b-a163-f9fdd81b9ef3-config-volume\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.976729 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.976538    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411592b1-9f45-453b-a163-f9fdd81b9ef3-tmp-dir\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.976729 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.976578    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zs7q\" (UniqueName: \"kubernetes.io/projected/411592b1-9f45-453b-a163-f9fdd81b9ef3-kube-api-access-4zs7q\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.976942 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.976919    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411592b1-9f45-453b-a163-f9fdd81b9ef3-tmp-dir\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.977007 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.976988    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/411592b1-9f45-453b-a163-f9fdd81b9ef3-config-volume\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.978731 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.978715    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411592b1-9f45-453b-a163-f9fdd81b9ef3-metrics-tls\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.984943 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.984925    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zs7q\" (UniqueName: \"kubernetes.io/projected/411592b1-9f45-453b-a163-f9fdd81b9ef3-kube-api-access-4zs7q\") pod \"dns-default-tnrlp\" (UID: \"411592b1-9f45-453b-a163-f9fdd81b9ef3\") " pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:15.991884 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:15.991864    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:16.002000 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.001981    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g48ff"
Jul 26 16:09:16.015121 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.015103    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t9gsq"
Jul 26 16:09:16.112077 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.112016    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:16.239009 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.238977    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b59467984-gbq79"]
Jul 26 16:09:16.243879 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:16.243852    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4464b0fb_0684_4a0c_b3c4_3cdbd643ffc8.slice/crio-dd558812a867298d6abae1202a726a18b0f3da81d5d4aa720703aa23dc485a9a WatchSource:0}: Error finding container dd558812a867298d6abae1202a726a18b0f3da81d5d4aa720703aa23dc485a9a: Status 404 returned error can't find the container with id dd558812a867298d6abae1202a726a18b0f3da81d5d4aa720703aa23dc485a9a
Jul 26 16:09:16.260024 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.259916    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t9gsq"]
Jul 26 16:09:16.262823 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:16.262787    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d1cceb_8043_4994_9bfe_8c690d05319e.slice/crio-5b564656d9f31772aa2cb31f253b33a080d12a765b7f3d7179c81a8bd576c648 WatchSource:0}: Error finding container 5b564656d9f31772aa2cb31f253b33a080d12a765b7f3d7179c81a8bd576c648: Status 404 returned error can't find the container with id 5b564656d9f31772aa2cb31f253b33a080d12a765b7f3d7179c81a8bd576c648
Jul 26 16:09:16.294274 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.294249    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:16.294274 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.294264    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:16.294451 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.294262    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:16.297627 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.297609    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 16:09:16.297730 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.297697    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h6wls\""
Jul 26 16:09:16.297779 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.297755    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 16:09:16.297814 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.297784    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 16:09:16.298761 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.298747    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6nt9n\""
Jul 26 16:09:16.298761 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.298756    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 16:09:16.306044 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.306022    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tnrlp"]
Jul 26 16:09:16.322126 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:16.322099    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod411592b1_9f45_453b_a163_f9fdd81b9ef3.slice/crio-57fdcce43aa92b7a9f793786be5dd4464589318c9a176460bc1fbd0328d65873 WatchSource:0}: Error finding container 57fdcce43aa92b7a9f793786be5dd4464589318c9a176460bc1fbd0328d65873: Status 404 returned error can't find the container with id 57fdcce43aa92b7a9f793786be5dd4464589318c9a176460bc1fbd0328d65873
Jul 26 16:09:16.448732 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.448702    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g48ff"]
Jul 26 16:09:16.451756 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:16.451731    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb089bff1_c417_441f_9921_521378315b57.slice/crio-0d4861e0c41bd76cba482cced4435e98bd07be8ddfe812b31de87a87bb66563f WatchSource:0}: Error finding container 0d4861e0c41bd76cba482cced4435e98bd07be8ddfe812b31de87a87bb66563f: Status 404 returned error can't find the container with id 0d4861e0c41bd76cba482cced4435e98bd07be8ddfe812b31de87a87bb66563f
Jul 26 16:09:16.724507 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.724420    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tnrlp" event={"ID":"411592b1-9f45-453b-a163-f9fdd81b9ef3","Type":"ContainerStarted","Data":"57fdcce43aa92b7a9f793786be5dd4464589318c9a176460bc1fbd0328d65873"}
Jul 26 16:09:16.725741 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.725681    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t9gsq" event={"ID":"85d1cceb-8043-4994-9bfe-8c690d05319e","Type":"ContainerStarted","Data":"5b564656d9f31772aa2cb31f253b33a080d12a765b7f3d7179c81a8bd576c648"}
Jul 26 16:09:16.727565 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.727542    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b59467984-gbq79" event={"ID":"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8","Type":"ContainerStarted","Data":"88227f289e75d49949f280a699f1f540db025e5c75331488c9c25396235e0225"}
Jul 26 16:09:16.727688 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.727587    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b59467984-gbq79" event={"ID":"4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8","Type":"ContainerStarted","Data":"dd558812a867298d6abae1202a726a18b0f3da81d5d4aa720703aa23dc485a9a"}
Jul 26 16:09:16.727993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.727836    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:16.728906 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.728878    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g48ff" event={"ID":"b089bff1-c417-441f-9921-521378315b57","Type":"ContainerStarted","Data":"0d4861e0c41bd76cba482cced4435e98bd07be8ddfe812b31de87a87bb66563f"}
Jul 26 16:09:16.748139 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:16.747513    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b59467984-gbq79" podStartSLOduration=2.747494278 podStartE2EDuration="2.747494278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:16.746699808 +0000 UTC m=+52.929784097" watchObservedRunningTime="2026-07-26 16:09:16.747494278 +0000 UTC m=+52.930578566"
Jul 26 16:09:23.749700 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:23.749656    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tnrlp" event={"ID":"411592b1-9f45-453b-a163-f9fdd81b9ef3","Type":"ContainerStarted","Data":"4fe656ecc66d0719146f20dbe84107003389bb535436c6cc8d34b149216729b9"}
Jul 26 16:09:23.750243 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:23.749753    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tnrlp" event={"ID":"411592b1-9f45-453b-a163-f9fdd81b9ef3","Type":"ContainerStarted","Data":"6acffe615e9d944cffe78d21024d03b22bbd14f068d65d412fe0fe20359d1fd5"}
Jul 26 16:09:23.750243 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:23.749943    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:23.751440 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:23.751409    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t9gsq" event={"ID":"85d1cceb-8043-4994-9bfe-8c690d05319e","Type":"ContainerStarted","Data":"9c4e3ec0b9f2ec1fb5810e4ff43dd12caddf0e1ac511a44dcb712902d2e741a1"}
Jul 26 16:09:23.753048 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:23.753025    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g48ff" event={"ID":"b089bff1-c417-441f-9921-521378315b57","Type":"ContainerStarted","Data":"b73b94252b7793134c742b160423f925a3147cd19e86758dda1dca618f574ddb"}
Jul 26 16:09:23.769002 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:23.768939    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tnrlp" podStartSLOduration=2.709823594 podStartE2EDuration="8.768918726s" totalImagesPullingTime="6.059095132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:23.766825641 +0000 UTC m=+59.949909928" watchObservedRunningTime="2026-07-26 16:09:23.768918726 +0000 UTC m=+59.952003013"
Jul 26 16:09:23.782152 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:23.782104    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t9gsq" podStartSLOduration=2.342317492 podStartE2EDuration="8.782088119s" totalImagesPullingTime="6.439770627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:23.781299039 +0000 UTC m=+59.964383329" watchObservedRunningTime="2026-07-26 16:09:23.782088119 +0000 UTC m=+59.965172406"
Jul 26 16:09:24.673673 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:24.673638    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn7k8"
Jul 26 16:09:26.765926 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:26.765883    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g48ff" event={"ID":"b089bff1-c417-441f-9921-521378315b57","Type":"ContainerStarted","Data":"bb0f31c5d98f7f426858ebc6d1a8da5f57b96c1e36c48776499ca331e9263f42"}
Jul 26 16:09:27.681217 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:27.681170    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj"]
Jul 26 16:09:27.683332 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:27.683303    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj"
Jul 26 16:09:27.686078 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:27.686054    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-q6lfc\""
Jul 26 16:09:27.686223 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:27.686053    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 26 16:09:27.692440 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:27.692380    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj"]
Jul 26 16:09:27.784249 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:27.784213    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6e170623-cf7e-4197-b340-47661c2a1597-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-gcptj\" (UID: \"6e170623-cf7e-4197-b340-47661c2a1597\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj"
Jul 26 16:09:27.885691 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:27.885653    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6e170623-cf7e-4197-b340-47661c2a1597-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-gcptj\" (UID: \"6e170623-cf7e-4197-b340-47661c2a1597\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj"
Jul 26 16:09:27.888340 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:27.888313    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6e170623-cf7e-4197-b340-47661c2a1597-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-gcptj\" (UID: \"6e170623-cf7e-4197-b340-47661c2a1597\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj"
Jul 26 16:09:27.994469 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:27.994429    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj"
Jul 26 16:09:28.995391 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:28.995349    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdb5s\" (UniqueName: \"kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s\") pod \"network-check-target-djgt6\" (UID: \"f7f5558e-066e-4005-9d86-3090d19d3d66\") " pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:28.998364 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:28.998331    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 16:09:29.008163 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.008135    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 16:09:29.018785 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.018764    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdb5s\" (UniqueName: \"kubernetes.io/projected/f7f5558e-066e-4005-9d86-3090d19d3d66-kube-api-access-tdb5s\") pod \"network-check-target-djgt6\" (UID: \"f7f5558e-066e-4005-9d86-3090d19d3d66\") " pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:29.095942 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.095910    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:29.095942 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.095954    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:29.099224 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.099182    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 16:09:29.099224 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.099210    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 16:09:29.108646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.108623    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15961897-ec82-41cf-8ce5-b036a30b987c-original-pull-secret\") pod \"global-pull-secret-syncer-5dfgr\" (UID: \"15961897-ec82-41cf-8ce5-b036a30b987c\") " pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:29.108761 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.108660    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d8d82fd-3ead-4428-896e-103c296da485-metrics-certs\") pod \"network-metrics-daemon-7xxbv\" (UID: \"9d8d82fd-3ead-4428-896e-103c296da485\") " pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:29.229245 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.229212    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h6wls\""
Jul 26 16:09:29.233721 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.233694    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6nt9n\""
Jul 26 16:09:29.236727 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.236707    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xxbv"
Jul 26 16:09:29.236854 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.236758    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dfgr"
Jul 26 16:09:29.241906 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.241886    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:29.719300 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.719267    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-djgt6"]
Jul 26 16:09:29.722142 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:29.722113    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f5558e_066e_4005_9d86_3090d19d3d66.slice/crio-683705aa6039a6d08c4b86b8d95217f76b7c183e9dd3182dd4f83a3ec091988f WatchSource:0}: Error finding container 683705aa6039a6d08c4b86b8d95217f76b7c183e9dd3182dd4f83a3ec091988f: Status 404 returned error can't find the container with id 683705aa6039a6d08c4b86b8d95217f76b7c183e9dd3182dd4f83a3ec091988f
Jul 26 16:09:29.776470 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.776434    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-djgt6" event={"ID":"f7f5558e-066e-4005-9d86-3090d19d3d66","Type":"ContainerStarted","Data":"683705aa6039a6d08c4b86b8d95217f76b7c183e9dd3182dd4f83a3ec091988f"}
Jul 26 16:09:29.778270 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.778245    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g48ff" event={"ID":"b089bff1-c417-441f-9921-521378315b57","Type":"ContainerStarted","Data":"b74a760df18a5c16102bb8be481e597707f16dd2180786937bfada4caf3633c6"}
Jul 26 16:09:29.815334 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.815291    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g48ff" podStartSLOduration=1.802148559 podStartE2EDuration="14.815276513s" totalImagesPullingTime="13.013127954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:09:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:29.807820543 +0000 UTC m=+65.990904830" watchObservedRunningTime="2026-07-26 16:09:29.815276513 +0000 UTC m=+65.998360792"
Jul 26 16:09:29.946521 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.946480    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7xxbv"]
Jul 26 16:09:29.947280 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.947228    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5dfgr"]
Jul 26 16:09:29.948092 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:29.948062    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj"]
Jul 26 16:09:29.950286 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:29.950264    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e170623_cf7e_4197_b340_47661c2a1597.slice/crio-34f8a17dc0ff3f49eb4ae22ac3f84297ef13bce9e556455ad3c90c652f88b1d0 WatchSource:0}: Error finding container 34f8a17dc0ff3f49eb4ae22ac3f84297ef13bce9e556455ad3c90c652f88b1d0: Status 404 returned error can't find the container with id 34f8a17dc0ff3f49eb4ae22ac3f84297ef13bce9e556455ad3c90c652f88b1d0
Jul 26 16:09:29.950846 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:29.950820    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8d82fd_3ead_4428_896e_103c296da485.slice/crio-1761875b1c8270f5d8b8a114b1e952fa257ec92dd6ce2b53a4d281d74ecc7d27 WatchSource:0}: Error finding container 1761875b1c8270f5d8b8a114b1e952fa257ec92dd6ce2b53a4d281d74ecc7d27: Status 404 returned error can't find the container with id 1761875b1c8270f5d8b8a114b1e952fa257ec92dd6ce2b53a4d281d74ecc7d27
Jul 26 16:09:29.951789 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:29.951768    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15961897_ec82_41cf_8ce5_b036a30b987c.slice/crio-1c745e9c5c87ca0f96bf056c15531854be2826bc6b99620c07bf5bf12e1590c4 WatchSource:0}: Error finding container 1c745e9c5c87ca0f96bf056c15531854be2826bc6b99620c07bf5bf12e1590c4: Status 404 returned error can't find the container with id 1c745e9c5c87ca0f96bf056c15531854be2826bc6b99620c07bf5bf12e1590c4
Jul 26 16:09:30.783831 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:30.783789    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xxbv" event={"ID":"9d8d82fd-3ead-4428-896e-103c296da485","Type":"ContainerStarted","Data":"1761875b1c8270f5d8b8a114b1e952fa257ec92dd6ce2b53a4d281d74ecc7d27"}
Jul 26 16:09:30.785721 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:30.785665    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5dfgr" event={"ID":"15961897-ec82-41cf-8ce5-b036a30b987c","Type":"ContainerStarted","Data":"1c745e9c5c87ca0f96bf056c15531854be2826bc6b99620c07bf5bf12e1590c4"}
Jul 26 16:09:30.787354 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:30.787293    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj" event={"ID":"6e170623-cf7e-4197-b340-47661c2a1597","Type":"ContainerStarted","Data":"34f8a17dc0ff3f49eb4ae22ac3f84297ef13bce9e556455ad3c90c652f88b1d0"}
Jul 26 16:09:31.792270 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:31.792221    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj" event={"ID":"6e170623-cf7e-4197-b340-47661c2a1597","Type":"ContainerStarted","Data":"a852fd88aee845e2fb5de01d10c008c3da051ae755372410776cde2491035b59"}
Jul 26 16:09:31.792709 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:31.792618    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj"
Jul 26 16:09:31.794338 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:31.794293    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xxbv" event={"ID":"9d8d82fd-3ead-4428-896e-103c296da485","Type":"ContainerStarted","Data":"e1b3e46e4f3f648cfe0763e797efdf9cdd09a816f2fad129dc872b6756c46a17"}
Jul 26 16:09:31.794462 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:31.794341    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xxbv" event={"ID":"9d8d82fd-3ead-4428-896e-103c296da485","Type":"ContainerStarted","Data":"6e8d94343d56626590f2c726d2c693d8e1be64844ecc2fda88ca3b152c10c3c6"}
Jul 26 16:09:31.798288 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:31.798267    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj"
Jul 26 16:09:31.807900 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:31.807857    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-gcptj" podStartSLOduration=3.319597767 podStartE2EDuration="4.807844457s" totalImagesPullingTime="1.48824669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:31.806654218 +0000 UTC m=+67.989738524" watchObservedRunningTime="2026-07-26 16:09:31.807844457 +0000 UTC m=+67.990928743"
Jul 26 16:09:31.838211 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:31.838135    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7xxbv" podStartSLOduration=66.351567077 podStartE2EDuration="1m7.838115067s" totalImagesPullingTime="1.48654799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:31.837413648 +0000 UTC m=+68.020497961" watchObservedRunningTime="2026-07-26 16:09:31.838115067 +0000 UTC m=+68.021199357"
Jul 26 16:09:32.745632 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.745600    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"]
Jul 26 16:09:32.750862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.750833    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.755202 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.755162    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 26 16:09:32.755328 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.755217    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 26 16:09:32.755328 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.755168    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 26 16:09:32.755328 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.755297    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 26 16:09:32.755328 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.755242    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wrw5n\""
Jul 26 16:09:32.755618 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.755162    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 26 16:09:32.761230 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.761203    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"]
Jul 26 16:09:32.825773 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.825738    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkgtp\" (UniqueName: \"kubernetes.io/projected/532db9ce-7850-4661-90e7-a6fd3d29dad5-kube-api-access-qkgtp\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.826171 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.825796    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.826171 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.825829    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.826171 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.825847    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/532db9ce-7850-4661-90e7-a6fd3d29dad5-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.927145 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.927099    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkgtp\" (UniqueName: \"kubernetes.io/projected/532db9ce-7850-4661-90e7-a6fd3d29dad5-kube-api-access-qkgtp\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.927343 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.927181    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.927343 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.927237    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.927343 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.927261    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/532db9ce-7850-4661-90e7-a6fd3d29dad5-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.928315 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:32.928292    2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 26 16:09:32.928411 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:32.928368    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-tls podName:532db9ce-7850-4661-90e7-a6fd3d29dad5 nodeName:}" failed. No retries permitted until 2026-07-26 16:09:33.428346146 +0000 UTC m=+69.611430424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-2zqnt" (UID: "532db9ce-7850-4661-90e7-a6fd3d29dad5") : secret "prometheus-operator-tls" not found
Jul 26 16:09:32.928995 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.928965    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/532db9ce-7850-4661-90e7-a6fd3d29dad5-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.931280 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.931257    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:32.937145 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:32.937102    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkgtp\" (UniqueName: \"kubernetes.io/projected/532db9ce-7850-4661-90e7-a6fd3d29dad5-kube-api-access-qkgtp\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:33.430799 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:33.430760    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:33.430986 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:33.430916    2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 26 16:09:33.430986 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:09:33.430982    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-tls podName:532db9ce-7850-4661-90e7-a6fd3d29dad5 nodeName:}" failed. No retries permitted until 2026-07-26 16:09:34.430966106 +0000 UTC m=+70.614050384 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-2zqnt" (UID: "532db9ce-7850-4661-90e7-a6fd3d29dad5") : secret "prometheus-operator-tls" not found
Jul 26 16:09:33.760337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:33.760303    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tnrlp"
Jul 26 16:09:34.439068 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:34.439035    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:34.441424 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:34.441405    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/532db9ce-7850-4661-90e7-a6fd3d29dad5-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-2zqnt\" (UID: \"532db9ce-7850-4661-90e7-a6fd3d29dad5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:34.561959 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:34.561935    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"
Jul 26 16:09:34.691681 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:34.691568    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-2zqnt"]
Jul 26 16:09:34.693955 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:34.693932    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532db9ce_7850_4661_90e7_a6fd3d29dad5.slice/crio-564c218646a2c955b06d904aa1da17e7014c0e9a0ef4a856533b3d53ba7491a2 WatchSource:0}: Error finding container 564c218646a2c955b06d904aa1da17e7014c0e9a0ef4a856533b3d53ba7491a2: Status 404 returned error can't find the container with id 564c218646a2c955b06d904aa1da17e7014c0e9a0ef4a856533b3d53ba7491a2
Jul 26 16:09:34.806731 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:34.806692    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt" event={"ID":"532db9ce-7850-4661-90e7-a6fd3d29dad5","Type":"ContainerStarted","Data":"564c218646a2c955b06d904aa1da17e7014c0e9a0ef4a856533b3d53ba7491a2"}
Jul 26 16:09:34.809052 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:34.809006    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5dfgr" event={"ID":"15961897-ec82-41cf-8ce5-b036a30b987c","Type":"ContainerStarted","Data":"7262b1350311f8ad45e2e213f69ab6e35a6df0fda8461aba8d64a3b4cf5a7efa"}
Jul 26 16:09:34.811026 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:34.810998    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-djgt6" event={"ID":"f7f5558e-066e-4005-9d86-3090d19d3d66","Type":"ContainerStarted","Data":"4f0362c941e38524c4696150b90ff8c2b781fbd5a872fe02c6609d1592c3f378"}
Jul 26 16:09:34.811271 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:34.811245    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:09:34.826363 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:34.826316    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5dfgr" podStartSLOduration=66.295727868 podStartE2EDuration="1m10.82629971s" totalImagesPullingTime="4.530571842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:34.825817832 +0000 UTC m=+71.008902120" watchObservedRunningTime="2026-07-26 16:09:34.82629971 +0000 UTC m=+71.009383974"
Jul 26 16:09:34.839763 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:34.839717    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-djgt6" podStartSLOduration=66.082246909 podStartE2EDuration="1m10.839701496s" totalImagesPullingTime="4.757454587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:34.839529628 +0000 UTC m=+71.022613933" watchObservedRunningTime="2026-07-26 16:09:34.839701496 +0000 UTC m=+71.022785782"
Jul 26 16:09:35.997126 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:35.997078    2570 patch_prober.go:28] interesting pod/image-registry-7b59467984-gbq79 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 26 16:09:35.997471 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:35.997155    2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7b59467984-gbq79" podUID="4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 16:09:36.819978 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:36.819941    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt" event={"ID":"532db9ce-7850-4661-90e7-a6fd3d29dad5","Type":"ContainerStarted","Data":"401e4efba1cee6cabd954818542beab67940322d8c4a5a93ecaa77527e30da70"}
Jul 26 16:09:36.820146 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:36.819999    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt" event={"ID":"532db9ce-7850-4661-90e7-a6fd3d29dad5","Type":"ContainerStarted","Data":"ded3e3371839b2f76bda51eb0c111a339f10e6c89e90bc2ba3f6bf2ffd2b4f2c"}
Jul 26 16:09:36.837489 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:36.837442    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-2zqnt" podStartSLOduration=3.644641221 podStartE2EDuration="4.837427657s" totalImagesPullingTime="1.192786436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:36.836873726 +0000 UTC m=+73.019958026" watchObservedRunningTime="2026-07-26 16:09:36.837427657 +0000 UTC m=+73.020511961"
Jul 26 16:09:37.736988 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:37.736960    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b59467984-gbq79"
Jul 26 16:09:39.107929 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.107895    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t4t5l"]
Jul 26 16:09:39.110324 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.110301    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.113361 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.113339    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 26 16:09:39.113477 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.113349    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 26 16:09:39.113576 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.113543    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wrthp\""
Jul 26 16:09:39.114541 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.114525    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 26 16:09:39.273102 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.273069    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5328de77-0706-4e80-b992-51512fa49017-metrics-client-ca\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.273295 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.273111    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47f2l\" (UniqueName: \"kubernetes.io/projected/5328de77-0706-4e80-b992-51512fa49017-kube-api-access-47f2l\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.273295 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.273152    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5328de77-0706-4e80-b992-51512fa49017-node-exporter-tls\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.273295 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.273176    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5328de77-0706-4e80-b992-51512fa49017-root\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.273295 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.273257    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5328de77-0706-4e80-b992-51512fa49017-node-exporter-textfile\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.273295 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.273289    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5328de77-0706-4e80-b992-51512fa49017-sys\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.273484 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.273309    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5328de77-0706-4e80-b992-51512fa49017-node-exporter-accelerators-collector-config\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.273484 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.273330    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5328de77-0706-4e80-b992-51512fa49017-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.273484 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.273351    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5328de77-0706-4e80-b992-51512fa49017-node-exporter-wtmp\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374277 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374171    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47f2l\" (UniqueName: \"kubernetes.io/projected/5328de77-0706-4e80-b992-51512fa49017-kube-api-access-47f2l\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374277 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374234    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5328de77-0706-4e80-b992-51512fa49017-node-exporter-tls\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374277 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374253    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5328de77-0706-4e80-b992-51512fa49017-root\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374277 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374269    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5328de77-0706-4e80-b992-51512fa49017-node-exporter-textfile\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374582 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374288    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5328de77-0706-4e80-b992-51512fa49017-sys\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374582 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374303    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5328de77-0706-4e80-b992-51512fa49017-node-exporter-accelerators-collector-config\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374582 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374321    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5328de77-0706-4e80-b992-51512fa49017-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374582 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374359    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5328de77-0706-4e80-b992-51512fa49017-root\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374582 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374371    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5328de77-0706-4e80-b992-51512fa49017-node-exporter-wtmp\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374582 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374461    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5328de77-0706-4e80-b992-51512fa49017-metrics-client-ca\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374582 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374486    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5328de77-0706-4e80-b992-51512fa49017-node-exporter-wtmp\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.374582 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374395    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5328de77-0706-4e80-b992-51512fa49017-sys\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.375002 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.374978    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5328de77-0706-4e80-b992-51512fa49017-node-exporter-textfile\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.375356 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.375336    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5328de77-0706-4e80-b992-51512fa49017-metrics-client-ca\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.375465 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.375445    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5328de77-0706-4e80-b992-51512fa49017-node-exporter-accelerators-collector-config\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.379761 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.379736    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5328de77-0706-4e80-b992-51512fa49017-node-exporter-tls\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.380752 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.380728    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5328de77-0706-4e80-b992-51512fa49017-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.382302 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.382280    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47f2l\" (UniqueName: \"kubernetes.io/projected/5328de77-0706-4e80-b992-51512fa49017-kube-api-access-47f2l\") pod \"node-exporter-t4t5l\" (UID: \"5328de77-0706-4e80-b992-51512fa49017\") " pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.418858 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.418836    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t4t5l"
Jul 26 16:09:39.428540 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:39.428519    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5328de77_0706_4e80_b992_51512fa49017.slice/crio-2574b0a2c0097ca8b1f4520938d008c1193220c2f3274f0392cb751c57d36f40 WatchSource:0}: Error finding container 2574b0a2c0097ca8b1f4520938d008c1193220c2f3274f0392cb751c57d36f40: Status 404 returned error can't find the container with id 2574b0a2c0097ca8b1f4520938d008c1193220c2f3274f0392cb751c57d36f40
Jul 26 16:09:39.831329 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:39.831297    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t4t5l" event={"ID":"5328de77-0706-4e80-b992-51512fa49017","Type":"ContainerStarted","Data":"2574b0a2c0097ca8b1f4520938d008c1193220c2f3274f0392cb751c57d36f40"}
Jul 26 16:09:40.836200 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:40.836153    2570 generic.go:356] "Generic (PLEG): container finished" podID="5328de77-0706-4e80-b992-51512fa49017" containerID="a99a017a5d0782e3086b58fd75f480f91dc13a12412f1f5604b54f7c2939c25e" exitCode=0
Jul 26 16:09:40.836568 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:40.836221    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t4t5l" event={"ID":"5328de77-0706-4e80-b992-51512fa49017","Type":"ContainerDied","Data":"a99a017a5d0782e3086b58fd75f480f91dc13a12412f1f5604b54f7c2939c25e"}
Jul 26 16:09:41.176848 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.176766    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"]
Jul 26 16:09:41.179119 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.179102    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.182058 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.182037    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 26 16:09:41.182350 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.182311    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 26 16:09:41.182350 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.182333    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 26 16:09:41.182559 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.182403    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 26 16:09:41.182559 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.182349    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-kv3ttv1nanfn\""
Jul 26 16:09:41.182559 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.182536    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-dqx7r\""
Jul 26 16:09:41.182713 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.182652    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 26 16:09:41.191382 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.191363    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"]
Jul 26 16:09:41.288625 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.288590    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.288625 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.288627    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2b70c7c-86b7-430d-8a6c-07122090e03d-metrics-client-ca\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.288816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.288657    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-tls\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.288816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.288731    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.288816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.288759    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw44z\" (UniqueName: \"kubernetes.io/projected/d2b70c7c-86b7-430d-8a6c-07122090e03d-kube-api-access-zw44z\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.288816 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.288793    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.288945 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.288861    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-grpc-tls\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.288945 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.288889    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.389559 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.389526    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.389706 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.389571    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-grpc-tls\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.389706 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.389593    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.389706 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.389612    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.389706 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.389631    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2b70c7c-86b7-430d-8a6c-07122090e03d-metrics-client-ca\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.389706 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.389661    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-tls\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.389706 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.389695    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.389979 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.389715    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw44z\" (UniqueName: \"kubernetes.io/projected/d2b70c7c-86b7-430d-8a6c-07122090e03d-kube-api-access-zw44z\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.390616 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.390559    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2b70c7c-86b7-430d-8a6c-07122090e03d-metrics-client-ca\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.392389 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.392354    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.392513 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.392401    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-grpc-tls\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.392568 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.392515    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.392861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.392840    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.392914 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.392900    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.392946 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.392928    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d2b70c7c-86b7-430d-8a6c-07122090e03d-secret-thanos-querier-tls\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.397290 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.397268    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw44z\" (UniqueName: \"kubernetes.io/projected/d2b70c7c-86b7-430d-8a6c-07122090e03d-kube-api-access-zw44z\") pod \"thanos-querier-66dbbcf55f-b8hqx\" (UID: \"d2b70c7c-86b7-430d-8a6c-07122090e03d\") " pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.487317 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.487290    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:41.622978 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.622840    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"]
Jul 26 16:09:41.625522 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:41.625487    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b70c7c_86b7_430d_8a6c_07122090e03d.slice/crio-ef9ab539604c515f0d57ef693aa70c2a6942a0750face5dc585c131d56430b6f WatchSource:0}: Error finding container ef9ab539604c515f0d57ef693aa70c2a6942a0750face5dc585c131d56430b6f: Status 404 returned error can't find the container with id ef9ab539604c515f0d57ef693aa70c2a6942a0750face5dc585c131d56430b6f
Jul 26 16:09:41.842347 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.842249    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t4t5l" event={"ID":"5328de77-0706-4e80-b992-51512fa49017","Type":"ContainerStarted","Data":"0cd29e56a57189d44d347a27f5d4e41ab9fdf20bf12195df5391b819c4031670"}
Jul 26 16:09:41.842347 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.842320    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t4t5l" event={"ID":"5328de77-0706-4e80-b992-51512fa49017","Type":"ContainerStarted","Data":"985f8f453841b56cbb662270714d8e4f3a641cab08f82f2d41e692f213b24622"}
Jul 26 16:09:41.843442 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.843420    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx" event={"ID":"d2b70c7c-86b7-430d-8a6c-07122090e03d","Type":"ContainerStarted","Data":"ef9ab539604c515f0d57ef693aa70c2a6942a0750face5dc585c131d56430b6f"}
Jul 26 16:09:41.860690 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:41.860650    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t4t5l" podStartSLOduration=1.864079713 podStartE2EDuration="2.860634844s" totalImagesPullingTime="932.989179ms" totalInitContainerRuntime="63.565952ms" isStatefulPod=true podCreationTimestamp="2026-07-26 16:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:41.859238776 +0000 UTC m=+78.042323085" watchObservedRunningTime="2026-07-26 16:09:41.860634844 +0000 UTC m=+78.043719131"
Jul 26 16:09:43.399159 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.399119    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-76b54c898d-8nvw2"]
Jul 26 16:09:43.401315 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.401293    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.405324 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.405296    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-z9brh\""
Jul 26 16:09:43.405324 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.405314    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7pkhfn0qmvnus\""
Jul 26 16:09:43.405324 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.405325    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 26 16:09:43.405536 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.405368    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 26 16:09:43.405536 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.405452    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 26 16:09:43.405681 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.405662    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 26 16:09:43.412797 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.412772    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76b54c898d-8nvw2"]
Jul 26 16:09:43.508049 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.508015    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-client-ca-bundle\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.508259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.508075    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-secret-metrics-server-client-certs\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.508259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.508101    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-secret-metrics-server-tls\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.508259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.508127    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-metrics-server-audit-profiles\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.508259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.508172    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-audit-log\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.508259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.508220    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.508259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.508244    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shhgd\" (UniqueName: \"kubernetes.io/projected/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-kube-api-access-shhgd\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.609364 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.609332    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-client-ca-bundle\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.609526 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.609378    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-secret-metrics-server-client-certs\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.609526 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.609396    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-secret-metrics-server-tls\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.609526 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.609514    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-metrics-server-audit-profiles\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.609688 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.609587    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-audit-log\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.609688 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.609627    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.609795 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.609759    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shhgd\" (UniqueName: \"kubernetes.io/projected/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-kube-api-access-shhgd\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.610146 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.610049    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-audit-log\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.610336 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.610318    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.610676 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.610652    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-metrics-server-audit-profiles\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.611836 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.611813    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-secret-metrics-server-tls\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.611933 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.611917    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-secret-metrics-server-client-certs\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.611984 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.611974    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-client-ca-bundle\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.618282 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.618261    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shhgd\" (UniqueName: \"kubernetes.io/projected/2c6ec0cf-f3ee-484a-94f7-81a0298e2beb-kube-api-access-shhgd\") pod \"metrics-server-76b54c898d-8nvw2\" (UID: \"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb\") " pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.713412 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.713381    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:09:43.854046 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.854018    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx" event={"ID":"d2b70c7c-86b7-430d-8a6c-07122090e03d","Type":"ContainerStarted","Data":"e4db030fb64e8cd45c0cb09d0d0b75592f74384b79f0d4dd99ded577293e0561"}
Jul 26 16:09:43.854134 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.854079    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx" event={"ID":"d2b70c7c-86b7-430d-8a6c-07122090e03d","Type":"ContainerStarted","Data":"2c662f6c2ff7bb48197a698a1ff3f61a82737b508415f76e771f06315a499bb8"}
Jul 26 16:09:43.854134 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.854106    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx" event={"ID":"d2b70c7c-86b7-430d-8a6c-07122090e03d","Type":"ContainerStarted","Data":"53303dfd2c767abf4cc3ca928065f83356265c8cf116456e26559138b513a7da"}
Jul 26 16:09:43.865675 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:43.865650    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76b54c898d-8nvw2"]
Jul 26 16:09:43.868622 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:43.868600    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6ec0cf_f3ee_484a_94f7_81a0298e2beb.slice/crio-fd5463afeb3871ac6d9868928a394b1a943b2285c1a40261a0c5259044c8c73e WatchSource:0}: Error finding container fd5463afeb3871ac6d9868928a394b1a943b2285c1a40261a0c5259044c8c73e: Status 404 returned error can't find the container with id fd5463afeb3871ac6d9868928a394b1a943b2285c1a40261a0c5259044c8c73e
Jul 26 16:09:44.859130 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:44.859095    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2" event={"ID":"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb","Type":"ContainerStarted","Data":"fd5463afeb3871ac6d9868928a394b1a943b2285c1a40261a0c5259044c8c73e"}
Jul 26 16:09:44.862175 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:44.862149    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx" event={"ID":"d2b70c7c-86b7-430d-8a6c-07122090e03d","Type":"ContainerStarted","Data":"871b4d663f151bb39e3565c0fcdf1ef70b470955d012609a720f892f627b469f"}
Jul 26 16:09:45.406199 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.406157    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 16:09:45.410516 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.410494    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.413502 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.413459    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 26 16:09:45.413617 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.413464    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 26 16:09:45.413700 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.413690    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 26 16:09:45.413983 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.413794    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hx6zg\""
Jul 26 16:09:45.413983 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.413910    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-afoesqngq9q5\""
Jul 26 16:09:45.414231 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.414081    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 26 16:09:45.414900 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.414858    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 26 16:09:45.415002 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.414956    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 26 16:09:45.415208 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.415155    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 26 16:09:45.415208 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.415170    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 26 16:09:45.415346 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.415276    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 26 16:09:45.415346 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.415173    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 26 16:09:45.415606 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.415590    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 26 16:09:45.415694 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.415669    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 26 16:09:45.423275 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.423257    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 26 16:09:45.424866 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.424844    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 26 16:09:45.431385 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.431361    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 16:09:45.527642 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527605    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.527642 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527642    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.527858 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527659    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.527858 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527711    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-config\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.527858 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527750    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.527858 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527770    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqr8k\" (UniqueName: \"kubernetes.io/projected/ca56160f-ab52-45c8-b99f-1f5f60723672-kube-api-access-cqr8k\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.527858 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527789    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.527858 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527806    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.527858 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527827    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-web-config\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527863    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ca56160f-ab52-45c8-b99f-1f5f60723672-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527895    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ca56160f-ab52-45c8-b99f-1f5f60723672-config-out\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527960    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527976    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.527998    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.528015    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.528033    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.528089    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.528110    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.528156    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.528221 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.528179    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628634    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628687    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628713    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628740    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-config\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628778    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628800    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqr8k\" (UniqueName: \"kubernetes.io/projected/ca56160f-ab52-45c8-b99f-1f5f60723672-kube-api-access-cqr8k\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628825    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628849    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628882    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-web-config\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628909    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ca56160f-ab52-45c8-b99f-1f5f60723672-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628932    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ca56160f-ab52-45c8-b99f-1f5f60723672-config-out\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.628977    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.629000    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.629025    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.629051    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.629078    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.630619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.629107    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.631570 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.629129    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.631570 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.629167    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.631570 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.629204    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.631570 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.630162    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.631570 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.631262    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.631570 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.631526    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.631570 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.631541    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.632554 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.632525    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.633086 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.633055    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.633691 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.633664    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.634056 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.634032    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ca56160f-ab52-45c8-b99f-1f5f60723672-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.634984 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.634927    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.636645 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.636620    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.636959 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.636922    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.637057 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.636968    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.637655 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.637583    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ca56160f-ab52-45c8-b99f-1f5f60723672-config-out\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.638373 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.638352    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ca56160f-ab52-45c8-b99f-1f5f60723672-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.639169 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.639150    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-config\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.639861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.639613    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.640666 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.640604    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.640666 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.640602    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.641623 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.641593    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ca56160f-ab52-45c8-b99f-1f5f60723672-web-config\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.642819 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.642797    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqr8k\" (UniqueName: \"kubernetes.io/projected/ca56160f-ab52-45c8-b99f-1f5f60723672-kube-api-access-cqr8k\") pod \"prometheus-k8s-0\" (UID: \"ca56160f-ab52-45c8-b99f-1f5f60723672\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.725850 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.725812    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:09:45.874471 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.874434    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx" event={"ID":"d2b70c7c-86b7-430d-8a6c-07122090e03d","Type":"ContainerStarted","Data":"01cbf61ed401ca2b4d0b816abfb65be0819cf9ea42f71d69e743352a8d87bc1d"}
Jul 26 16:09:45.874935 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.874507    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx" event={"ID":"d2b70c7c-86b7-430d-8a6c-07122090e03d","Type":"ContainerStarted","Data":"71073b62cd6a19e97864b7c36e8a86e44916466278bdc289db608e04cffc5453"}
Jul 26 16:09:45.874935 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.874605    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:45.876612 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.876586    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2" event={"ID":"2c6ec0cf-f3ee-484a-94f7-81a0298e2beb","Type":"ContainerStarted","Data":"a4b2fbb883262b2471d0ffd89bde67312b5f485cbf898f32474cee228f41c583"}
Jul 26 16:09:45.887765 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.887688    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 16:09:45.893517 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:09:45.893491    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca56160f_ab52_45c8_b99f_1f5f60723672.slice/crio-a1c18800589bd68658a83fc858749ae970177b01495d6bb1d798eae9a6e78e3f WatchSource:0}: Error finding container a1c18800589bd68658a83fc858749ae970177b01495d6bb1d798eae9a6e78e3f: Status 404 returned error can't find the container with id a1c18800589bd68658a83fc858749ae970177b01495d6bb1d798eae9a6e78e3f
Jul 26 16:09:45.902495 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.902453    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx" podStartSLOduration=1.965874238 podStartE2EDuration="4.902435847s" totalImagesPullingTime="2.936561609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:45.90137244 +0000 UTC m=+82.084456746" watchObservedRunningTime="2026-07-26 16:09:45.902435847 +0000 UTC m=+82.085520125"
Jul 26 16:09:45.927112 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:45.927064    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2" podStartSLOduration=1.43931253 podStartE2EDuration="2.927051145s" totalImagesPullingTime="1.487738615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:45.926086203 +0000 UTC m=+82.109170502" watchObservedRunningTime="2026-07-26 16:09:45.927051145 +0000 UTC m=+82.110135431"
Jul 26 16:09:46.881727 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:46.881617    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ca56160f-ab52-45c8-b99f-1f5f60723672","Type":"ContainerStarted","Data":"a1c18800589bd68658a83fc858749ae970177b01495d6bb1d798eae9a6e78e3f"}
Jul 26 16:09:47.887861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:47.887819    2570 generic.go:356] "Generic (PLEG): container finished" podID="ca56160f-ab52-45c8-b99f-1f5f60723672" containerID="4a672733d834cd4464c3e65001ac81f3b82ff920abecdeb46e062b0ba6e25d32" exitCode=0
Jul 26 16:09:47.888269 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:47.887885    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ca56160f-ab52-45c8-b99f-1f5f60723672","Type":"ContainerDied","Data":"4a672733d834cd4464c3e65001ac81f3b82ff920abecdeb46e062b0ba6e25d32"}
Jul 26 16:09:51.889300 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:51.889278    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66dbbcf55f-b8hqx"
Jul 26 16:09:51.911550 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:51.911508    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ca56160f-ab52-45c8-b99f-1f5f60723672","Type":"ContainerStarted","Data":"bc6909c9e4474cfb587bf34800ebb17b5fb0b3e0ae3416def0364ad71bd9116e"}
Jul 26 16:09:51.911669 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:51.911604    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ca56160f-ab52-45c8-b99f-1f5f60723672","Type":"ContainerStarted","Data":"e2dbdae93f2b52c61383e54031b0ea51bc995b455418b7c74c3c9e58df10636b"}
Jul 26 16:09:52.918918 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:52.918879    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ca56160f-ab52-45c8-b99f-1f5f60723672","Type":"ContainerStarted","Data":"cd5cfc65148b5a02f72d01f6ef7eac62b6d2f8f29ac3a6d184c1afcc0570b126"}
Jul 26 16:09:52.919340 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:52.919001    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ca56160f-ab52-45c8-b99f-1f5f60723672","Type":"ContainerStarted","Data":"09403ad4484719f8534982fb23ea9088b6f29a8ed22a07af4a61202974eeb9f5"}
Jul 26 16:09:52.919340 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:52.919046    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ca56160f-ab52-45c8-b99f-1f5f60723672","Type":"ContainerStarted","Data":"5dc101dc9efd35519c21e782498520c4a3322c101dabb24229d45e3184962774"}
Jul 26 16:09:52.919340 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:52.919088    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ca56160f-ab52-45c8-b99f-1f5f60723672","Type":"ContainerStarted","Data":"cdae091094076fa1a9bab3a2f89cfc959acc0cfc32db8eecfb49ee61cf64199d"}
Jul 26 16:09:52.951246 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:52.951175    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.159205058 podStartE2EDuration="7.951158011s" totalImagesPullingTime="4.773353644s" totalInitContainerRuntime="18.599309ms" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:52.94866441 +0000 UTC m=+89.131748718" watchObservedRunningTime="2026-07-26 16:09:52.951158011 +0000 UTC m=+89.134242300"
Jul 26 16:09:55.726454 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:09:55.726416    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:10:03.713979 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:10:03.713944    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:10:03.714429 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:10:03.714036    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:10:05.817673 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:10:05.817639    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-djgt6"
Jul 26 16:10:23.718757 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:10:23.718725    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:10:23.722701 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:10:23.722669    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-76b54c898d-8nvw2"
Jul 26 16:10:45.726810 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:10:45.726751    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:10:45.745932 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:10:45.745909    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:10:46.123949 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:10:46.123878    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 16:13:24.179664 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:24.179628    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:13:24.184367 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:24.184337    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:13:24.185744 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:24.185724    2570 kubelet.go:1648] "Image garbage collection succeeded"
Jul 26 16:13:33.960687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:33.960654    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"]
Jul 26 16:13:33.964004 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:33.963990    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"
Jul 26 16:13:33.966716 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:33.966688    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 26 16:13:33.966716 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:33.966698    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-qkb8j\""
Jul 26 16:13:33.967932 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:33.967911    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 26 16:13:33.973317 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:33.973291    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"]
Jul 26 16:13:34.048226 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:34.048166    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2hk4\" (UniqueName: \"kubernetes.io/projected/18e4cbe3-ab70-4a6e-85f3-3de5ab613a02-kube-api-access-k2hk4\") pod \"openshift-lws-operator-fd8ccff4c-hlzx6\" (UID: \"18e4cbe3-ab70-4a6e-85f3-3de5ab613a02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"
Jul 26 16:13:34.048398 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:34.048240    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18e4cbe3-ab70-4a6e-85f3-3de5ab613a02-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hlzx6\" (UID: \"18e4cbe3-ab70-4a6e-85f3-3de5ab613a02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"
Jul 26 16:13:34.148797 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:34.148762    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2hk4\" (UniqueName: \"kubernetes.io/projected/18e4cbe3-ab70-4a6e-85f3-3de5ab613a02-kube-api-access-k2hk4\") pod \"openshift-lws-operator-fd8ccff4c-hlzx6\" (UID: \"18e4cbe3-ab70-4a6e-85f3-3de5ab613a02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"
Jul 26 16:13:34.148965 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:34.148810    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18e4cbe3-ab70-4a6e-85f3-3de5ab613a02-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hlzx6\" (UID: \"18e4cbe3-ab70-4a6e-85f3-3de5ab613a02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"
Jul 26 16:13:34.149139 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:34.149124    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18e4cbe3-ab70-4a6e-85f3-3de5ab613a02-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hlzx6\" (UID: \"18e4cbe3-ab70-4a6e-85f3-3de5ab613a02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"
Jul 26 16:13:34.157532 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:34.157504    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2hk4\" (UniqueName: \"kubernetes.io/projected/18e4cbe3-ab70-4a6e-85f3-3de5ab613a02-kube-api-access-k2hk4\") pod \"openshift-lws-operator-fd8ccff4c-hlzx6\" (UID: \"18e4cbe3-ab70-4a6e-85f3-3de5ab613a02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"
Jul 26 16:13:34.284453 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:34.284384    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"
Jul 26 16:13:34.412677 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:34.412653    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6"]
Jul 26 16:13:34.415156 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:13:34.415133    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18e4cbe3_ab70_4a6e_85f3_3de5ab613a02.slice/crio-535ce2e2b59f70dcec2b0cfa1e51efd18ff8563e232433a5ae02227f44c95898 WatchSource:0}: Error finding container 535ce2e2b59f70dcec2b0cfa1e51efd18ff8563e232433a5ae02227f44c95898: Status 404 returned error can't find the container with id 535ce2e2b59f70dcec2b0cfa1e51efd18ff8563e232433a5ae02227f44c95898
Jul 26 16:13:34.416687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:34.416670    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:13:34.714691 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:34.714654    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6" event={"ID":"18e4cbe3-ab70-4a6e-85f3-3de5ab613a02","Type":"ContainerStarted","Data":"535ce2e2b59f70dcec2b0cfa1e51efd18ff8563e232433a5ae02227f44c95898"}
Jul 26 16:13:37.730880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:37.730845    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6" event={"ID":"18e4cbe3-ab70-4a6e-85f3-3de5ab613a02","Type":"ContainerStarted","Data":"fcf9d7cf7710177d898d10a14b74c92201ecb671f28e056d6fecaecc668d0539"}
Jul 26 16:13:37.748257 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:37.748206    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlzx6" podStartSLOduration=2.234766368 podStartE2EDuration="4.748176977s" totalImagesPullingTime="2.513410609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:13:37.746589985 +0000 UTC m=+313.929674272" watchObservedRunningTime="2026-07-26 16:13:37.748176977 +0000 UTC m=+313.931261265"
Jul 26 16:13:54.129377 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.129344    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"]
Jul 26 16:13:54.132596 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.132577    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.135655 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.135637    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lg5vj\""
Jul 26 16:13:54.136245 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.136210    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 26 16:13:54.136413 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.136251    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 26 16:13:54.136413 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.136253    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 26 16:13:54.136413 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.136355    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 26 16:13:54.160152 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.160124    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"]
Jul 26 16:13:54.219700 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.219662    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.219870 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.219706    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvmrh\" (UniqueName: \"kubernetes.io/projected/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-kube-api-access-zvmrh\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.219870 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.219749    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-manifests\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.219870 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.219821    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-tmp\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.219994 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.219894    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.320419 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.320386    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-manifests\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.320657 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.320464    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-tmp\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.320657 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.320526    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.320829 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.320756    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.320829 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.320787    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvmrh\" (UniqueName: \"kubernetes.io/projected/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-kube-api-access-zvmrh\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.320964 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.320938    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-manifests\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.323361 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.323334    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-tmp\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.323833 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.323809    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.324161 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.324141    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.331296 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.331272    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvmrh\" (UniqueName: \"kubernetes.io/projected/1ff7b7b9-0c36-4233-aadb-1fe983e267e2-kube-api-access-zvmrh\") pod \"opendatahub-operator-controller-manager-7bd4986cdb-kz98g\" (UID: \"1ff7b7b9-0c36-4233-aadb-1fe983e267e2\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.442394 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.442318    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:13:54.466380 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.466347    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"]
Jul 26 16:13:54.469790 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.469767    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.472971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.472950    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 26 16:13:54.473100 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.472951    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 26 16:13:54.473243 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.473230    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 26 16:13:54.473298 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.473283    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-87vm7\""
Jul 26 16:13:54.485560 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.485540    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"]
Jul 26 16:13:54.522167 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.522141    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbbsk\" (UniqueName: \"kubernetes.io/projected/cfd9f872-0925-4839-8790-9453c11c1a63-kube-api-access-xbbsk\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.522327 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.522174    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfd9f872-0925-4839-8790-9453c11c1a63-metrics-cert\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.522327 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.522220    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cfd9f872-0925-4839-8790-9453c11c1a63-manager-config\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.522327 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.522279    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfd9f872-0925-4839-8790-9453c11c1a63-cert\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.598080 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.598050    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"]
Jul 26 16:13:54.601298 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:13:54.601270    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ff7b7b9_0c36_4233_aadb_1fe983e267e2.slice/crio-c4bcebef9857dd69d36236fef25e9c9b3e4435689300824b947b9e043535e86d WatchSource:0}: Error finding container c4bcebef9857dd69d36236fef25e9c9b3e4435689300824b947b9e043535e86d: Status 404 returned error can't find the container with id c4bcebef9857dd69d36236fef25e9c9b3e4435689300824b947b9e043535e86d
Jul 26 16:13:54.623524 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.623499    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbbsk\" (UniqueName: \"kubernetes.io/projected/cfd9f872-0925-4839-8790-9453c11c1a63-kube-api-access-xbbsk\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.623641 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.623531    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfd9f872-0925-4839-8790-9453c11c1a63-metrics-cert\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.623641 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.623554    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cfd9f872-0925-4839-8790-9453c11c1a63-manager-config\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.623641 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.623582    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfd9f872-0925-4839-8790-9453c11c1a63-cert\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.624292 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.624271    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cfd9f872-0925-4839-8790-9453c11c1a63-manager-config\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.625924 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.625905    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfd9f872-0925-4839-8790-9453c11c1a63-cert\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.626022 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.626005    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfd9f872-0925-4839-8790-9453c11c1a63-metrics-cert\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.635087 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.635066    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbbsk\" (UniqueName: \"kubernetes.io/projected/cfd9f872-0925-4839-8790-9453c11c1a63-kube-api-access-xbbsk\") pod \"lws-controller-manager-76c849bdf9-jsslf\" (UID: \"cfd9f872-0925-4839-8790-9453c11c1a63\") " pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.778971 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.778937    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:54.800164 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.800133    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g" event={"ID":"1ff7b7b9-0c36-4233-aadb-1fe983e267e2","Type":"ContainerStarted","Data":"c4bcebef9857dd69d36236fef25e9c9b3e4435689300824b947b9e043535e86d"}
Jul 26 16:13:54.911446 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:54.911294    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"]
Jul 26 16:13:54.913886 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:13:54.913861    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd9f872_0925_4839_8790_9453c11c1a63.slice/crio-b89c4f5a72ed895f9266fae0e73c3d2f47210427bc7f2d8118d4f0c38e8e95d9 WatchSource:0}: Error finding container b89c4f5a72ed895f9266fae0e73c3d2f47210427bc7f2d8118d4f0c38e8e95d9: Status 404 returned error can't find the container with id b89c4f5a72ed895f9266fae0e73c3d2f47210427bc7f2d8118d4f0c38e8e95d9
Jul 26 16:13:55.807655 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:55.807619    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf" event={"ID":"cfd9f872-0925-4839-8790-9453c11c1a63","Type":"ContainerStarted","Data":"b89c4f5a72ed895f9266fae0e73c3d2f47210427bc7f2d8118d4f0c38e8e95d9"}
Jul 26 16:13:58.821732 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:58.821687    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g" event={"ID":"1ff7b7b9-0c36-4233-aadb-1fe983e267e2","Type":"ContainerStarted","Data":"e2edda5712dfe5b575f4dc14961200fc338c56bb423ab95e5ff3bbb6d530fc52"}
Jul 26 16:13:58.823155 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:58.823129    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf" event={"ID":"cfd9f872-0925-4839-8790-9453c11c1a63","Type":"ContainerStarted","Data":"b52f51b069e52c23e73b57211cdb925a00662ff1a1ce65e0ea2a216fbb80c9a9"}
Jul 26 16:13:58.823305 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:58.823291    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:13:58.862072 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:58.862024    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf" podStartSLOduration=1.719364928 podStartE2EDuration="4.86200777s" totalImagesPullingTime="3.142642842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:13:58.86055141 +0000 UTC m=+335.043635698" watchObservedRunningTime="2026-07-26 16:13:58.86200777 +0000 UTC m=+335.045092058"
Jul 26 16:13:59.828917 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:59.828868    2570 generic.go:356] "Generic (PLEG): container finished" podID="1ff7b7b9-0c36-4233-aadb-1fe983e267e2" containerID="e2edda5712dfe5b575f4dc14961200fc338c56bb423ab95e5ff3bbb6d530fc52" exitCode=0
Jul 26 16:13:59.829305 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:13:59.828945    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g" event={"ID":"1ff7b7b9-0c36-4233-aadb-1fe983e267e2","Type":"ContainerDied","Data":"e2edda5712dfe5b575f4dc14961200fc338c56bb423ab95e5ff3bbb6d530fc52"}
Jul 26 16:14:00.834501 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:00.834465    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g" event={"ID":"1ff7b7b9-0c36-4233-aadb-1fe983e267e2","Type":"ContainerStarted","Data":"78c980b2fd7c2411c59af9729a9bd389d3ae592676984fde14f3a02b15aab487"}
Jul 26 16:14:00.835003 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:00.834600    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:14:00.861156 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:00.861101    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g" podStartSLOduration=2.085609207 podStartE2EDuration="6.861085171s" totalImagesPullingTime="3.721082338s" totalInitContainerRuntime="1.054393626s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:13:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:14:00.86027158 +0000 UTC m=+337.043355885" watchObservedRunningTime="2026-07-26 16:14:00.861085171 +0000 UTC m=+337.044169458"
Jul 26 16:14:09.831263 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:09.831235    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76c849bdf9-jsslf"
Jul 26 16:14:11.840783 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:11.840756    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bd4986cdb-kz98g"
Jul 26 16:14:14.159362 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.159325    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"]
Jul 26 16:14:14.167787 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.167762    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.171204 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.171172    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-k82ck\""
Jul 26 16:14:14.172309 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.172279    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 26 16:14:14.172561 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.172545    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 26 16:14:14.172646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.172626    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 26 16:14:14.172746 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.172688    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 26 16:14:14.180157 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.180135    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"]
Jul 26 16:14:14.290549 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.290519    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9t4k\" (UniqueName: \"kubernetes.io/projected/1c823567-6693-49d6-bed4-96472ee97dec-kube-api-access-p9t4k\") pod \"kube-auth-proxy-6ff4cb76d8-lbtrd\" (UID: \"1c823567-6693-49d6-bed4-96472ee97dec\") " pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.290704 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.290573    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c823567-6693-49d6-bed4-96472ee97dec-tls-certs\") pod \"kube-auth-proxy-6ff4cb76d8-lbtrd\" (UID: \"1c823567-6693-49d6-bed4-96472ee97dec\") " pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.290704 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.290595    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c823567-6693-49d6-bed4-96472ee97dec-tmp\") pod \"kube-auth-proxy-6ff4cb76d8-lbtrd\" (UID: \"1c823567-6693-49d6-bed4-96472ee97dec\") " pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.391967 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.391933    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9t4k\" (UniqueName: \"kubernetes.io/projected/1c823567-6693-49d6-bed4-96472ee97dec-kube-api-access-p9t4k\") pod \"kube-auth-proxy-6ff4cb76d8-lbtrd\" (UID: \"1c823567-6693-49d6-bed4-96472ee97dec\") " pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.392146 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.391987    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c823567-6693-49d6-bed4-96472ee97dec-tls-certs\") pod \"kube-auth-proxy-6ff4cb76d8-lbtrd\" (UID: \"1c823567-6693-49d6-bed4-96472ee97dec\") " pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.392146 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.392005    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c823567-6693-49d6-bed4-96472ee97dec-tmp\") pod \"kube-auth-proxy-6ff4cb76d8-lbtrd\" (UID: \"1c823567-6693-49d6-bed4-96472ee97dec\") " pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.394108 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.394087    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c823567-6693-49d6-bed4-96472ee97dec-tmp\") pod \"kube-auth-proxy-6ff4cb76d8-lbtrd\" (UID: \"1c823567-6693-49d6-bed4-96472ee97dec\") " pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.394418 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.394398    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c823567-6693-49d6-bed4-96472ee97dec-tls-certs\") pod \"kube-auth-proxy-6ff4cb76d8-lbtrd\" (UID: \"1c823567-6693-49d6-bed4-96472ee97dec\") " pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.399808 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.399782    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9t4k\" (UniqueName: \"kubernetes.io/projected/1c823567-6693-49d6-bed4-96472ee97dec-kube-api-access-p9t4k\") pod \"kube-auth-proxy-6ff4cb76d8-lbtrd\" (UID: \"1c823567-6693-49d6-bed4-96472ee97dec\") " pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.477588 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.477552    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"
Jul 26 16:14:14.609164 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.609136    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd"]
Jul 26 16:14:14.611748 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:14:14.611723    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c823567_6693_49d6_bed4_96472ee97dec.slice/crio-5f4b6580f52e6afd7ad2ef47d0da00de966bdc4572a5887d48e1cc6a39e2f087 WatchSource:0}: Error finding container 5f4b6580f52e6afd7ad2ef47d0da00de966bdc4572a5887d48e1cc6a39e2f087: Status 404 returned error can't find the container with id 5f4b6580f52e6afd7ad2ef47d0da00de966bdc4572a5887d48e1cc6a39e2f087
Jul 26 16:14:14.890409 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:14.890329    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd" event={"ID":"1c823567-6693-49d6-bed4-96472ee97dec","Type":"ContainerStarted","Data":"5f4b6580f52e6afd7ad2ef47d0da00de966bdc4572a5887d48e1cc6a39e2f087"}
Jul 26 16:14:17.904015 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:17.903977    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd" event={"ID":"1c823567-6693-49d6-bed4-96472ee97dec","Type":"ContainerStarted","Data":"36957901f5a95f1aeb4eed96833e880bb256c84152ff0688920bf59bc9d3391d"}
Jul 26 16:14:17.922952 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:17.922899    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6ff4cb76d8-lbtrd" podStartSLOduration=1.457061545 podStartE2EDuration="3.922883097s" totalImagesPullingTime="2.465821552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:14:17.921804922 +0000 UTC m=+354.104889223" watchObservedRunningTime="2026-07-26 16:14:17.922883097 +0000 UTC m=+354.105967383"
Jul 26 16:14:24.281384 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.281348    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"]
Jul 26 16:14:24.286251 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.286229    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:14:24.290503 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.290297    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-bsdmd\""
Jul 26 16:14:24.291652 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.291630    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 26 16:14:24.302254 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.302232    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"]
Jul 26 16:14:24.378474 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.378436    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d9486caa-a9f7-4f10-a6af-2a64e508313c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xt7ng\" (UID: \"d9486caa-a9f7-4f10-a6af-2a64e508313c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:14:24.378474 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.378477    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm4hn\" (UniqueName: \"kubernetes.io/projected/d9486caa-a9f7-4f10-a6af-2a64e508313c-kube-api-access-vm4hn\") pod \"llmisvc-controller-manager-777c6bd7d5-xt7ng\" (UID: \"d9486caa-a9f7-4f10-a6af-2a64e508313c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:14:24.478986 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.478944    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d9486caa-a9f7-4f10-a6af-2a64e508313c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xt7ng\" (UID: \"d9486caa-a9f7-4f10-a6af-2a64e508313c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:14:24.478986 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.478986    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm4hn\" (UniqueName: \"kubernetes.io/projected/d9486caa-a9f7-4f10-a6af-2a64e508313c-kube-api-access-vm4hn\") pod \"llmisvc-controller-manager-777c6bd7d5-xt7ng\" (UID: \"d9486caa-a9f7-4f10-a6af-2a64e508313c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:14:24.479222 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:14:24.479097    2570 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 26 16:14:24.479222 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:14:24.479168    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9486caa-a9f7-4f10-a6af-2a64e508313c-cert podName:d9486caa-a9f7-4f10-a6af-2a64e508313c nodeName:}" failed. No retries permitted until 2026-07-26 16:14:24.979150871 +0000 UTC m=+361.162235140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d9486caa-a9f7-4f10-a6af-2a64e508313c-cert") pod "llmisvc-controller-manager-777c6bd7d5-xt7ng" (UID: "d9486caa-a9f7-4f10-a6af-2a64e508313c") : secret "llmisvc-webhook-server-cert" not found
Jul 26 16:14:24.489890 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.489857    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm4hn\" (UniqueName: \"kubernetes.io/projected/d9486caa-a9f7-4f10-a6af-2a64e508313c-kube-api-access-vm4hn\") pod \"llmisvc-controller-manager-777c6bd7d5-xt7ng\" (UID: \"d9486caa-a9f7-4f10-a6af-2a64e508313c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:14:24.983335 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.983305    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d9486caa-a9f7-4f10-a6af-2a64e508313c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xt7ng\" (UID: \"d9486caa-a9f7-4f10-a6af-2a64e508313c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:14:24.985813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:24.985778    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d9486caa-a9f7-4f10-a6af-2a64e508313c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xt7ng\" (UID: \"d9486caa-a9f7-4f10-a6af-2a64e508313c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:14:25.195574 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:25.195526    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:14:25.334683 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:25.334654    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"]
Jul 26 16:14:25.336129 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:14:25.336086    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd9486caa_a9f7_4f10_a6af_2a64e508313c.slice/crio-ca93e42ba89703391909dc51f19c31642263193fbe98649b918ca5cf147e9aaa WatchSource:0}: Error finding container ca93e42ba89703391909dc51f19c31642263193fbe98649b918ca5cf147e9aaa: Status 404 returned error can't find the container with id ca93e42ba89703391909dc51f19c31642263193fbe98649b918ca5cf147e9aaa
Jul 26 16:14:25.942135 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:25.942098    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng" event={"ID":"d9486caa-a9f7-4f10-a6af-2a64e508313c","Type":"ContainerStarted","Data":"ca93e42ba89703391909dc51f19c31642263193fbe98649b918ca5cf147e9aaa"}
Jul 26 16:14:28.958147 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:28.958111    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng" event={"ID":"d9486caa-a9f7-4f10-a6af-2a64e508313c","Type":"ContainerStarted","Data":"bb273f159565ef26c264327c7d8f0b4b8f187507319e3285aafea1667e607226"}
Jul 26 16:14:28.958556 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:28.958228    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:14:28.974907 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:28.974863    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng" podStartSLOduration=1.9370838400000001 podStartE2EDuration="4.974848132s" totalImagesPullingTime="3.037764292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:14:28.973484809 +0000 UTC m=+365.156569127" watchObservedRunningTime="2026-07-26 16:14:28.974848132 +0000 UTC m=+365.157932419"
Jul 26 16:14:59.965676 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:14:59.965639    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xt7ng"
Jul 26 16:16:48.748077 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.748044    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:16:48.751277 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.751259    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"
Jul 26 16:16:48.754006 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.753988    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 26 16:16:48.754111 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.754007    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 26 16:16:48.755293 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.755277    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5kmdp\""
Jul 26 16:16:48.755351 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.755283    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 26 16:16:48.760455 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.760435    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:16:48.789684 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.789661    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:16:48.873153 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.873126    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw4c9\" (UniqueName: \"kubernetes.io/projected/22668da6-cd04-4a5c-9c24-1a33de4b9f55-kube-api-access-zw4c9\") pod \"limitador-limitador-54d6d988d5-b6nbh\" (UID: \"22668da6-cd04-4a5c-9c24-1a33de4b9f55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"
Jul 26 16:16:48.873319 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.873169    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/22668da6-cd04-4a5c-9c24-1a33de4b9f55-config-file\") pod \"limitador-limitador-54d6d988d5-b6nbh\" (UID: \"22668da6-cd04-4a5c-9c24-1a33de4b9f55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"
Jul 26 16:16:48.973493 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.973463    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/22668da6-cd04-4a5c-9c24-1a33de4b9f55-config-file\") pod \"limitador-limitador-54d6d988d5-b6nbh\" (UID: \"22668da6-cd04-4a5c-9c24-1a33de4b9f55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"
Jul 26 16:16:48.973639 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.973523    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw4c9\" (UniqueName: \"kubernetes.io/projected/22668da6-cd04-4a5c-9c24-1a33de4b9f55-kube-api-access-zw4c9\") pod \"limitador-limitador-54d6d988d5-b6nbh\" (UID: \"22668da6-cd04-4a5c-9c24-1a33de4b9f55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"
Jul 26 16:16:48.974043 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.974026    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/22668da6-cd04-4a5c-9c24-1a33de4b9f55-config-file\") pod \"limitador-limitador-54d6d988d5-b6nbh\" (UID: \"22668da6-cd04-4a5c-9c24-1a33de4b9f55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"
Jul 26 16:16:48.983052 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:48.983030    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw4c9\" (UniqueName: \"kubernetes.io/projected/22668da6-cd04-4a5c-9c24-1a33de4b9f55-kube-api-access-zw4c9\") pod \"limitador-limitador-54d6d988d5-b6nbh\" (UID: \"22668da6-cd04-4a5c-9c24-1a33de4b9f55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"
Jul 26 16:16:49.061555 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.061460    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"
Jul 26 16:16:49.186347 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.186252    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:16:49.189183 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:16:49.189151    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22668da6_cd04_4a5c_9c24_1a33de4b9f55.slice/crio-992e207ab73f17fbd0a89145c72653c6e198b184414f3409027a355a4802c994 WatchSource:0}: Error finding container 992e207ab73f17fbd0a89145c72653c6e198b184414f3409027a355a4802c994: Status 404 returned error can't find the container with id 992e207ab73f17fbd0a89145c72653c6e198b184414f3409027a355a4802c994
Jul 26 16:16:49.196728 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.196702    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6nx9m"]
Jul 26 16:16:49.201046 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.201029    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6nx9m"
Jul 26 16:16:49.203802 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.203782    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-krgkv\""
Jul 26 16:16:49.206647 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.206627    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6nx9m"]
Jul 26 16:16:49.276867 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.276836    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csf5z\" (UniqueName: \"kubernetes.io/projected/2d4d6357-97d9-4a0f-bf69-44869a3cca87-kube-api-access-csf5z\") pod \"authorino-58cd955675-6nx9m\" (UID: \"2d4d6357-97d9-4a0f-bf69-44869a3cca87\") " pod="kuadrant-system/authorino-58cd955675-6nx9m"
Jul 26 16:16:49.378110 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.378038    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csf5z\" (UniqueName: \"kubernetes.io/projected/2d4d6357-97d9-4a0f-bf69-44869a3cca87-kube-api-access-csf5z\") pod \"authorino-58cd955675-6nx9m\" (UID: \"2d4d6357-97d9-4a0f-bf69-44869a3cca87\") " pod="kuadrant-system/authorino-58cd955675-6nx9m"
Jul 26 16:16:49.386621 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.386593    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csf5z\" (UniqueName: \"kubernetes.io/projected/2d4d6357-97d9-4a0f-bf69-44869a3cca87-kube-api-access-csf5z\") pod \"authorino-58cd955675-6nx9m\" (UID: \"2d4d6357-97d9-4a0f-bf69-44869a3cca87\") " pod="kuadrant-system/authorino-58cd955675-6nx9m"
Jul 26 16:16:49.511757 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.511727    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6nx9m"
Jul 26 16:16:49.535150 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.535123    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh" event={"ID":"22668da6-cd04-4a5c-9c24-1a33de4b9f55","Type":"ContainerStarted","Data":"992e207ab73f17fbd0a89145c72653c6e198b184414f3409027a355a4802c994"}
Jul 26 16:16:49.637620 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:49.637595    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6nx9m"]
Jul 26 16:16:49.639489 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:16:49.639465    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d4d6357_97d9_4a0f_bf69_44869a3cca87.slice/crio-8a3fc38bd5cf9db73728080057ff14d289c79fbe25893281827f3faf7bc302e4 WatchSource:0}: Error finding container 8a3fc38bd5cf9db73728080057ff14d289c79fbe25893281827f3faf7bc302e4: Status 404 returned error can't find the container with id 8a3fc38bd5cf9db73728080057ff14d289c79fbe25893281827f3faf7bc302e4
Jul 26 16:16:50.542921 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:50.542883    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6nx9m" event={"ID":"2d4d6357-97d9-4a0f-bf69-44869a3cca87","Type":"ContainerStarted","Data":"8a3fc38bd5cf9db73728080057ff14d289c79fbe25893281827f3faf7bc302e4"}
Jul 26 16:16:54.491321 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:54.487966    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6nx9m"]
Jul 26 16:16:54.563055 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:54.563019    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6nx9m" event={"ID":"2d4d6357-97d9-4a0f-bf69-44869a3cca87","Type":"ContainerStarted","Data":"47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96"}
Jul 26 16:16:54.564527 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:54.564503    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh" event={"ID":"22668da6-cd04-4a5c-9c24-1a33de4b9f55","Type":"ContainerStarted","Data":"3c8979953715d1ddfb1a221bc54c7e842f4a72c65a16a8f003c7e62606fbda7d"}
Jul 26 16:16:54.564720 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:54.564587    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"
Jul 26 16:16:54.597364 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:54.597311    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6nx9m" podStartSLOduration=1.66472096 podStartE2EDuration="5.597294832s" totalImagesPullingTime="3.932573872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:16:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:16:54.596058687 +0000 UTC m=+510.779142974" watchObservedRunningTime="2026-07-26 16:16:54.597294832 +0000 UTC m=+510.780379097"
Jul 26 16:16:54.622208 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:54.622137    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh" podStartSLOduration=2.241302784 podStartE2EDuration="6.622120911s" totalImagesPullingTime="4.380818127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:16:54.620894111 +0000 UTC m=+510.803978408" watchObservedRunningTime="2026-07-26 16:16:54.622120911 +0000 UTC m=+510.805205234"
Jul 26 16:16:55.568640 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:55.568604    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6nx9m" podUID="2d4d6357-97d9-4a0f-bf69-44869a3cca87" containerName="authorino" containerID="cri-o://47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96" gracePeriod=30
Jul 26 16:16:55.817494 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:55.817473    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6nx9m"
Jul 26 16:16:55.935632 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:55.935546    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csf5z\" (UniqueName: \"kubernetes.io/projected/2d4d6357-97d9-4a0f-bf69-44869a3cca87-kube-api-access-csf5z\") pod \"2d4d6357-97d9-4a0f-bf69-44869a3cca87\" (UID: \"2d4d6357-97d9-4a0f-bf69-44869a3cca87\") "
Jul 26 16:16:55.937669 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:55.937647    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4d6357-97d9-4a0f-bf69-44869a3cca87-kube-api-access-csf5z" (OuterVolumeSpecName: "kube-api-access-csf5z") pod "2d4d6357-97d9-4a0f-bf69-44869a3cca87" (UID: "2d4d6357-97d9-4a0f-bf69-44869a3cca87"). InnerVolumeSpecName "kube-api-access-csf5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:16:56.036509 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:56.036482    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csf5z\" (UniqueName: \"kubernetes.io/projected/2d4d6357-97d9-4a0f-bf69-44869a3cca87-kube-api-access-csf5z\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:16:56.574289 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:56.574206    2570 generic.go:356] "Generic (PLEG): container finished" podID="2d4d6357-97d9-4a0f-bf69-44869a3cca87" containerID="47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96" exitCode=0
Jul 26 16:16:56.574289 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:56.574247    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6nx9m"
Jul 26 16:16:56.574289 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:56.574279    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6nx9m" event={"ID":"2d4d6357-97d9-4a0f-bf69-44869a3cca87","Type":"ContainerDied","Data":"47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96"}
Jul 26 16:16:56.574773 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:56.574317    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6nx9m" event={"ID":"2d4d6357-97d9-4a0f-bf69-44869a3cca87","Type":"ContainerDied","Data":"8a3fc38bd5cf9db73728080057ff14d289c79fbe25893281827f3faf7bc302e4"}
Jul 26 16:16:56.574773 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:56.574340    2570 scope.go:117] "RemoveContainer" containerID="47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96"
Jul 26 16:16:56.582927 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:56.582910    2570 scope.go:117] "RemoveContainer" containerID="47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96"
Jul 26 16:16:56.583154 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:16:56.583135    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96\": container with ID starting with 47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96 not found: ID does not exist" containerID="47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96"
Jul 26 16:16:56.583223 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:56.583162    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96"} err="failed to get container status \"47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96\": rpc error: code = NotFound desc = could not find container \"47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96\": container with ID starting with 47d9a92994b891ce315ff86dc92101e2c314a341de526b1a7f78ef5667077f96 not found: ID does not exist"
Jul 26 16:16:56.590639 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:56.590612    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6nx9m"]
Jul 26 16:16:56.596139 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:56.596120    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6nx9m"]
Jul 26 16:16:58.299625 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:16:58.299590    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d4d6357-97d9-4a0f-bf69-44869a3cca87" path="/var/lib/kubelet/pods/2d4d6357-97d9-4a0f-bf69-44869a3cca87/volumes"
Jul 26 16:17:05.570184 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:17:05.570153    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"
Jul 26 16:18:06.840077 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:06.840038    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:18:24.206863 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:24.206832    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:18:24.208644 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:24.208622    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:18:27.274527 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:27.274487    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:18:40.137394 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.137364    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"]
Jul 26 16:18:40.139965 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.137683    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d4d6357-97d9-4a0f-bf69-44869a3cca87" containerName="authorino"
Jul 26 16:18:40.139965 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.137694    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d4d6357-97d9-4a0f-bf69-44869a3cca87" containerName="authorino"
Jul 26 16:18:40.139965 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.137742    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d4d6357-97d9-4a0f-bf69-44869a3cca87" containerName="authorino"
Jul 26 16:18:40.140568 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.140553    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.143513 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.143486    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 26 16:18:40.143646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.143487    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 26 16:18:40.145072 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.145052    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 26 16:18:40.145169 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.145088    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hhbrk\""
Jul 26 16:18:40.163050 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.163019    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"]
Jul 26 16:18:40.215539 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.215505    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvpxp\" (UniqueName: \"kubernetes.io/projected/49b021e8-5109-443c-9304-e9bfc4ba9365-kube-api-access-mvpxp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.215698 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.215555    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.215698 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.215574    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.215698 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.215591    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.215698 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.215661    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b021e8-5109-443c-9304-e9bfc4ba9365-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.215863 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.215708    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.316175 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.316144    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.316175 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.316176    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.316434 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.316211    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.316434 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.316244    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b021e8-5109-443c-9304-e9bfc4ba9365-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.316434 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.316268    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.316434 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.316314    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvpxp\" (UniqueName: \"kubernetes.io/projected/49b021e8-5109-443c-9304-e9bfc4ba9365-kube-api-access-mvpxp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.316632 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.316547    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.316632 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.316611    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.316715 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.316695    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.318478 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.318457    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b021e8-5109-443c-9304-e9bfc4ba9365-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.318635 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.318617    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b021e8-5109-443c-9304-e9bfc4ba9365-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.324444 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.324417    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvpxp\" (UniqueName: \"kubernetes.io/projected/49b021e8-5109-443c-9304-e9bfc4ba9365-kube-api-access-mvpxp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f6scz\" (UID: \"49b021e8-5109-443c-9304-e9bfc4ba9365\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.450753 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.450724    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:40.580404 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.580377    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"]
Jul 26 16:18:40.582983 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:18:40.582954    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b021e8_5109_443c_9304_e9bfc4ba9365.slice/crio-a54c0133d705b049a7e6c29241c520459546fa4af234c34cb8731ec0e4fcf4b6 WatchSource:0}: Error finding container a54c0133d705b049a7e6c29241c520459546fa4af234c34cb8731ec0e4fcf4b6: Status 404 returned error can't find the container with id a54c0133d705b049a7e6c29241c520459546fa4af234c34cb8731ec0e4fcf4b6
Jul 26 16:18:40.584386 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.584364    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:18:40.813830 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:40.813740    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:18:41.015549 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:41.015514    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz" event={"ID":"49b021e8-5109-443c-9304-e9bfc4ba9365","Type":"ContainerStarted","Data":"a54c0133d705b049a7e6c29241c520459546fa4af234c34cb8731ec0e4fcf4b6"}
Jul 26 16:18:42.022049 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:42.022015    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz" event={"ID":"49b021e8-5109-443c-9304-e9bfc4ba9365","Type":"ContainerStarted","Data":"2fb2e78009ff6021cfcd1d9bd7181f2c7ccdab76b9eb1de1f56c128a2f7a9d9a"}
Jul 26 16:18:42.040388 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:42.040343    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz" podStartSLOduration=1.049376949 podStartE2EDuration="2.040328932s" totalImagesPullingTime="990.951983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:18:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:18:42.038638359 +0000 UTC m=+618.221722646" watchObservedRunningTime="2026-07-26 16:18:42.040328932 +0000 UTC m=+618.223413219"
Jul 26 16:18:50.451321 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:50.451232    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:50.451321 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:50.451295    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:50.464074 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:50.464047    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:51.071263 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:51.071230    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f6scz"
Jul 26 16:18:55.221626 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:55.221590    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:18:59.713009 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:18:59.712971    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:19:04.345335 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.345304    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"]
Jul 26 16:19:04.348676 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.348658    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.351290 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.351271    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 26 16:19:04.357086 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.357067    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"]
Jul 26 16:19:04.524496 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.524460    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtx4p\" (UniqueName: \"kubernetes.io/projected/8657640b-1cdb-40fc-b83d-5898af353b1c-kube-api-access-gtx4p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.524662 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.524511    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.524662 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.524569    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.524662 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.524605    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.524662 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.524627    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8657640b-1cdb-40fc-b83d-5898af353b1c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.524662 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.524655    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.625667 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.625581    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtx4p\" (UniqueName: \"kubernetes.io/projected/8657640b-1cdb-40fc-b83d-5898af353b1c-kube-api-access-gtx4p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.625667 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.625639    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.625667 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.625663    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.625900 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.625682    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.625900 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.625700    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8657640b-1cdb-40fc-b83d-5898af353b1c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.625900 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.625724    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.626144 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.626115    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.626273 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.626145    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.626273 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.626163    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.628035 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.628014    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8657640b-1cdb-40fc-b83d-5898af353b1c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.628342 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.628321    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8657640b-1cdb-40fc-b83d-5898af353b1c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.633543 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.633524    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtx4p\" (UniqueName: \"kubernetes.io/projected/8657640b-1cdb-40fc-b83d-5898af353b1c-kube-api-access-gtx4p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9xc64\" (UID: \"8657640b-1cdb-40fc-b83d-5898af353b1c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.659693 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.659660    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:04.786668 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:04.786619    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"]
Jul 26 16:19:04.789619 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:19:04.789588    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8657640b_1cdb_40fc_b83d_5898af353b1c.slice/crio-17cffb1f64d8dadb4a52f155cef59ae795d64cd398eb0d428faa1b2c2c306d3e WatchSource:0}: Error finding container 17cffb1f64d8dadb4a52f155cef59ae795d64cd398eb0d428faa1b2c2c306d3e: Status 404 returned error can't find the container with id 17cffb1f64d8dadb4a52f155cef59ae795d64cd398eb0d428faa1b2c2c306d3e
Jul 26 16:19:05.118733 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:05.118697    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64" event={"ID":"8657640b-1cdb-40fc-b83d-5898af353b1c","Type":"ContainerStarted","Data":"5be6941bb6c2608f3299acb9431a7712c787a8e9f2f9ba6c5949e0094e92f00a"}
Jul 26 16:19:05.118941 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:05.118753    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64" event={"ID":"8657640b-1cdb-40fc-b83d-5898af353b1c","Type":"ContainerStarted","Data":"17cffb1f64d8dadb4a52f155cef59ae795d64cd398eb0d428faa1b2c2c306d3e"}
Jul 26 16:19:05.161628 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:05.161578    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64" podStartSLOduration=0.987837391 podStartE2EDuration="1.161562495s" totalImagesPullingTime="173.725104ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:19:05.158881143 +0000 UTC m=+641.341965430" watchObservedRunningTime="2026-07-26 16:19:05.161562495 +0000 UTC m=+641.344646782"
Jul 26 16:19:06.913915 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:06.913880    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:19:14.659799 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:14.659763    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:14.660210 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:14.659849    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:14.672401 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:14.672376    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:15.176769 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:15.176737    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9xc64"
Jul 26 16:19:16.835752 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:16.835718    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"]
Jul 26 16:19:16.842382 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:16.842360    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:16.846121 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:16.846099    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 26 16:19:16.846510 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:16.846492    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"]
Jul 26 16:19:16.940913 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:16.940881    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:16.941065 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:16.940932    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:16.941065 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:16.940993    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppqwv\" (UniqueName: \"kubernetes.io/projected/97b0301b-3fe5-494a-8820-9ee5fbc18277-kube-api-access-ppqwv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:16.941065 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:16.941038    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:16.941065 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:16.941054    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97b0301b-3fe5-494a-8820-9ee5fbc18277-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:16.941224 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:16.941143    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.041788 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.041745    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.041943 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.041800    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.041943 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.041845    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.041943 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.041877    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppqwv\" (UniqueName: \"kubernetes.io/projected/97b0301b-3fe5-494a-8820-9ee5fbc18277-kube-api-access-ppqwv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.041943 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.041911    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.041943 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.041934    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97b0301b-3fe5-494a-8820-9ee5fbc18277-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.042253 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.042229    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.042305 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.042282    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.042349 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.042293    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.044305 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.044283    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97b0301b-3fe5-494a-8820-9ee5fbc18277-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.044533 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.044513    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97b0301b-3fe5-494a-8820-9ee5fbc18277-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.052591 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.052569    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppqwv\" (UniqueName: \"kubernetes.io/projected/97b0301b-3fe5-494a-8820-9ee5fbc18277-kube-api-access-ppqwv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-q86lp\" (UID: \"97b0301b-3fe5-494a-8820-9ee5fbc18277\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.152634 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.152553    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:17.284434 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:17.284412    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"]
Jul 26 16:19:17.286429 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:19:17.286401    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97b0301b_3fe5_494a_8820_9ee5fbc18277.slice/crio-dafb04bcf7510604205ad980aa481a300ab1f47299c5a4da387857fbcb62721b WatchSource:0}: Error finding container dafb04bcf7510604205ad980aa481a300ab1f47299c5a4da387857fbcb62721b: Status 404 returned error can't find the container with id dafb04bcf7510604205ad980aa481a300ab1f47299c5a4da387857fbcb62721b
Jul 26 16:19:18.180269 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:18.180232    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp" event={"ID":"97b0301b-3fe5-494a-8820-9ee5fbc18277","Type":"ContainerStarted","Data":"397a8b488ffab40f9d94f44396106ef0cdd3a9514b203f1a47c4374088e593ce"}
Jul 26 16:19:18.180648 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:18.180276    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp" event={"ID":"97b0301b-3fe5-494a-8820-9ee5fbc18277","Type":"ContainerStarted","Data":"dafb04bcf7510604205ad980aa481a300ab1f47299c5a4da387857fbcb62721b"}
Jul 26 16:19:18.198070 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:18.198019    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp" podStartSLOduration=2.024416244 podStartE2EDuration="2.198002738s" totalImagesPullingTime="173.586494ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:19:18.197031754 +0000 UTC m=+654.380116043" watchObservedRunningTime="2026-07-26 16:19:18.198002738 +0000 UTC m=+654.381087025"
Jul 26 16:19:19.111285 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:19.111250    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:19:27.153436 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:27.153401    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:27.153904 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:27.153523    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:27.165983 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:27.165957    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:19:27.234540 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:19:27.234506    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-q86lp"
Jul 26 16:21:00.555392 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.555352    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d795fc685-jbzp6"]
Jul 26 16:21:00.558717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.558701    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d795fc685-jbzp6"
Jul 26 16:21:00.562870 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.562851    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 26 16:21:00.562974 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.562852    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-krgkv\""
Jul 26 16:21:00.575787 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.575760    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d795fc685-jbzp6"]
Jul 26 16:21:00.622713 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.622680    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-945ml\" (UniqueName: \"kubernetes.io/projected/a4521ccb-7dc2-43e9-98d4-3e646f571cf1-kube-api-access-945ml\") pod \"authorino-7d795fc685-jbzp6\" (UID: \"a4521ccb-7dc2-43e9-98d4-3e646f571cf1\") " pod="kuadrant-system/authorino-7d795fc685-jbzp6"
Jul 26 16:21:00.622882 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.622815    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4521ccb-7dc2-43e9-98d4-3e646f571cf1-tls-cert\") pod \"authorino-7d795fc685-jbzp6\" (UID: \"a4521ccb-7dc2-43e9-98d4-3e646f571cf1\") " pod="kuadrant-system/authorino-7d795fc685-jbzp6"
Jul 26 16:21:00.724202 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.724166    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-945ml\" (UniqueName: \"kubernetes.io/projected/a4521ccb-7dc2-43e9-98d4-3e646f571cf1-kube-api-access-945ml\") pod \"authorino-7d795fc685-jbzp6\" (UID: \"a4521ccb-7dc2-43e9-98d4-3e646f571cf1\") " pod="kuadrant-system/authorino-7d795fc685-jbzp6"
Jul 26 16:21:00.724359 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.724274    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4521ccb-7dc2-43e9-98d4-3e646f571cf1-tls-cert\") pod \"authorino-7d795fc685-jbzp6\" (UID: \"a4521ccb-7dc2-43e9-98d4-3e646f571cf1\") " pod="kuadrant-system/authorino-7d795fc685-jbzp6"
Jul 26 16:21:00.726689 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.726666    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4521ccb-7dc2-43e9-98d4-3e646f571cf1-tls-cert\") pod \"authorino-7d795fc685-jbzp6\" (UID: \"a4521ccb-7dc2-43e9-98d4-3e646f571cf1\") " pod="kuadrant-system/authorino-7d795fc685-jbzp6"
Jul 26 16:21:00.732145 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.732123    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-945ml\" (UniqueName: \"kubernetes.io/projected/a4521ccb-7dc2-43e9-98d4-3e646f571cf1-kube-api-access-945ml\") pod \"authorino-7d795fc685-jbzp6\" (UID: \"a4521ccb-7dc2-43e9-98d4-3e646f571cf1\") " pod="kuadrant-system/authorino-7d795fc685-jbzp6"
Jul 26 16:21:00.868041 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.867932    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d795fc685-jbzp6"
Jul 26 16:21:00.996093 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:00.996067    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d795fc685-jbzp6"]
Jul 26 16:21:00.998482 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:21:00.998455    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4521ccb_7dc2_43e9_98d4_3e646f571cf1.slice/crio-ad4c77cd88547da66f606e2712460819ca9e1e7be6cf50025536c2d8e857b508 WatchSource:0}: Error finding container ad4c77cd88547da66f606e2712460819ca9e1e7be6cf50025536c2d8e857b508: Status 404 returned error can't find the container with id ad4c77cd88547da66f606e2712460819ca9e1e7be6cf50025536c2d8e857b508
Jul 26 16:21:01.630436 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:01.630400    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d795fc685-jbzp6" event={"ID":"a4521ccb-7dc2-43e9-98d4-3e646f571cf1","Type":"ContainerStarted","Data":"6e0b45677adc93b3f2837c9145b51254ca013b67c708f3479dc2e09ccf8afd96"}
Jul 26 16:21:01.630436 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:01.630440    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d795fc685-jbzp6" event={"ID":"a4521ccb-7dc2-43e9-98d4-3e646f571cf1","Type":"ContainerStarted","Data":"ad4c77cd88547da66f606e2712460819ca9e1e7be6cf50025536c2d8e857b508"}
Jul 26 16:21:01.651491 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:01.651441    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d795fc685-jbzp6" podStartSLOduration=1.226722627 podStartE2EDuration="1.65142132s" totalImagesPullingTime="424.698693ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:21:01.649660952 +0000 UTC m=+757.832745239" watchObservedRunningTime="2026-07-26 16:21:01.65142132 +0000 UTC m=+757.834505606"
Jul 26 16:21:34.113239 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:34.113202    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:21:43.511574 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:43.511534    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:21:54.011738 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:21:54.011653    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:23:24.234818 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:23:24.234735    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:23:24.235407 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:23:24.234855    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:23:35.518882 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:23:35.518841    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:23:43.813455 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:23:43.813411    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:24:13.816813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:24:13.816775    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:24:23.014252 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:24:23.014216    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:24:31.815774 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:24:31.815736    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:24:38.005899 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:24:38.005867    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:24:48.412228 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:24:48.412139    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:24:58.927717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:24:58.927680    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:25:09.415094 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:25:09.415055    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:25:19.420483 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:25:19.420448    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:25:42.014597 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:25:42.014563    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:25:51.216307 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:25:51.216272    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:26:15.722265 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:26:15.722230    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:26:26.410560 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:26:26.410486    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:26:41.116660 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:26:41.116629    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:26:50.920213 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:26:50.920154    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:27:00.624151 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:27:00.624110    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:27:09.917260 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:27:09.917227    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:27:18.814917 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:27:18.814880    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:28:24.261017 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:28:24.260989    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:28:24.261775 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:28:24.261756    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:29:06.225202 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:29:06.225150    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:29:17.143704 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:29:17.143627    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:29:25.809885 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:29:25.809850    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:29:38.220925 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:29:38.220890    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:29:49.426990 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:29:49.426954    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:29:58.816737 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:29:58.816700    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:30:09.023343 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:30:09.023310    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:30:19.314456 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:30:19.314420    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:30:29.932964 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:30:29.932928    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:30:39.319462 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:30:39.319427    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:30:49.924813 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:30:49.924724    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:31:11.817517 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:11.817482    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:31:20.911380 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:20.911337    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:31:32.135547 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:32.135510    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:31:34.040745 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.040705    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"]
Jul 26 16:31:34.044049 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.044020    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.046949 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.046927    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-aad57859-kserve-self-signed-certs\""
Jul 26 16:31:34.054014 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.053990    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"]
Jul 26 16:31:34.175709 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.175674    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-dshm\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.175883 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.175714    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-tmp-dir\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.175883 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.175742    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3656ad-df83-4419-8aef-bdb26af88585-tls-certs\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.175883 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.175773    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-model-cache\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.175883 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.175869    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-home\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.176033 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.175939    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2nkw\" (UniqueName: \"kubernetes.io/projected/ed3656ad-df83-4419-8aef-bdb26af88585-kube-api-access-z2nkw\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.277099 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.277055    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2nkw\" (UniqueName: \"kubernetes.io/projected/ed3656ad-df83-4419-8aef-bdb26af88585-kube-api-access-z2nkw\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.277409 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.277385    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-dshm\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.277549 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.277533    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-tmp-dir\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.277646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.277633    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3656ad-df83-4419-8aef-bdb26af88585-tls-certs\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.277751 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.277735    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-model-cache\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.277894 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.277879    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-home\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.278398 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.278377    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-home\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.278779 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.278757    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-tmp-dir\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.279758 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.279736    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-model-cache\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.281942 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.281921    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-dshm\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.282818 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.282791    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3656ad-df83-4419-8aef-bdb26af88585-tls-certs\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.292868 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.292803    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2nkw\" (UniqueName: \"kubernetes.io/projected/ed3656ad-df83-4419-8aef-bdb26af88585-kube-api-access-z2nkw\") pod \"e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") " pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.354341 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.354299    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:34.495172 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:34.495146    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"]
Jul 26 16:31:34.498317 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:31:34.498275    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded3656ad_df83_4419_8aef_bdb26af88585.slice/crio-42634639995b62c86a948f9fa2593a0fa8e56a5c7dc9549f2da18e0d357ac1c4 WatchSource:0}: Error finding container 42634639995b62c86a948f9fa2593a0fa8e56a5c7dc9549f2da18e0d357ac1c4: Status 404 returned error can't find the container with id 42634639995b62c86a948f9fa2593a0fa8e56a5c7dc9549f2da18e0d357ac1c4
Jul 26 16:31:35.401673 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:35.401623    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp" event={"ID":"ed3656ad-df83-4419-8aef-bdb26af88585","Type":"ContainerStarted","Data":"ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8"}
Jul 26 16:31:35.401673 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:35.401671    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp" event={"ID":"ed3656ad-df83-4419-8aef-bdb26af88585","Type":"ContainerStarted","Data":"42634639995b62c86a948f9fa2593a0fa8e56a5c7dc9549f2da18e0d357ac1c4"}
Jul 26 16:31:35.420765 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:35.420715    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp" podStartSLOduration=1.420699897 podStartE2EDuration="1.420699897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:31:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:31:35.420004842 +0000 UTC m=+1391.603089130" watchObservedRunningTime="2026-07-26 16:31:35.420699897 +0000 UTC m=+1391.603784183"
Jul 26 16:31:44.354689 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:44.354653    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:44.354689 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:44.354700    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:44.367330 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:44.367308    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:44.454725 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:44.454692    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:31:44.814691 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:31:44.814659    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:32:03.750921 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:03.750883    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"]
Jul 26 16:32:06.517664 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:06.517631    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:32:18.770210 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:18.770084    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp" podUID="ed3656ad-df83-4419-8aef-bdb26af88585" containerName="main" containerID="cri-o://ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8" gracePeriod=45
Jul 26 16:32:19.014082 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.014058    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:32:19.087263 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087171    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-model-cache\") pod \"ed3656ad-df83-4419-8aef-bdb26af88585\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") "
Jul 26 16:32:19.087263 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087252    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-home\") pod \"ed3656ad-df83-4419-8aef-bdb26af88585\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") "
Jul 26 16:32:19.087453 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087273    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2nkw\" (UniqueName: \"kubernetes.io/projected/ed3656ad-df83-4419-8aef-bdb26af88585-kube-api-access-z2nkw\") pod \"ed3656ad-df83-4419-8aef-bdb26af88585\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") "
Jul 26 16:32:19.087453 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087330    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3656ad-df83-4419-8aef-bdb26af88585-tls-certs\") pod \"ed3656ad-df83-4419-8aef-bdb26af88585\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") "
Jul 26 16:32:19.087453 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087362    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-dshm\") pod \"ed3656ad-df83-4419-8aef-bdb26af88585\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") "
Jul 26 16:32:19.087453 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087389    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-tmp-dir\") pod \"ed3656ad-df83-4419-8aef-bdb26af88585\" (UID: \"ed3656ad-df83-4419-8aef-bdb26af88585\") "
Jul 26 16:32:19.087453 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087388    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed3656ad-df83-4419-8aef-bdb26af88585" (UID: "ed3656ad-df83-4419-8aef-bdb26af88585"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:32:19.087711 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087503    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-home" (OuterVolumeSpecName: "home") pod "ed3656ad-df83-4419-8aef-bdb26af88585" (UID: "ed3656ad-df83-4419-8aef-bdb26af88585"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:32:19.087711 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087674    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed3656ad-df83-4419-8aef-bdb26af88585" (UID: "ed3656ad-df83-4419-8aef-bdb26af88585"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:32:19.087711 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087675    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:32:19.087829 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.087714    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:32:19.089523 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.089493    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3656ad-df83-4419-8aef-bdb26af88585-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed3656ad-df83-4419-8aef-bdb26af88585" (UID: "ed3656ad-df83-4419-8aef-bdb26af88585"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:32:19.089637 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.089524    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-dshm" (OuterVolumeSpecName: "dshm") pod "ed3656ad-df83-4419-8aef-bdb26af88585" (UID: "ed3656ad-df83-4419-8aef-bdb26af88585"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:32:19.089637 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.089524    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed3656ad-df83-4419-8aef-bdb26af88585-kube-api-access-z2nkw" (OuterVolumeSpecName: "kube-api-access-z2nkw") pod "ed3656ad-df83-4419-8aef-bdb26af88585" (UID: "ed3656ad-df83-4419-8aef-bdb26af88585"). InnerVolumeSpecName "kube-api-access-z2nkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:32:19.188546 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.188507    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:32:19.188546 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.188538    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2nkw\" (UniqueName: \"kubernetes.io/projected/ed3656ad-df83-4419-8aef-bdb26af88585-kube-api-access-z2nkw\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:32:19.188546 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.188547    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3656ad-df83-4419-8aef-bdb26af88585-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:32:19.188546 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.188556    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3656ad-df83-4419-8aef-bdb26af88585-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:32:19.597028 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.596992    2570 generic.go:356] "Generic (PLEG): container finished" podID="ed3656ad-df83-4419-8aef-bdb26af88585" containerID="ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8" exitCode=0
Jul 26 16:32:19.597028 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.597032    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"
Jul 26 16:32:19.597312 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.597045    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp" event={"ID":"ed3656ad-df83-4419-8aef-bdb26af88585","Type":"ContainerDied","Data":"ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8"}
Jul 26 16:32:19.597312 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.597084    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp" event={"ID":"ed3656ad-df83-4419-8aef-bdb26af88585","Type":"ContainerDied","Data":"42634639995b62c86a948f9fa2593a0fa8e56a5c7dc9549f2da18e0d357ac1c4"}
Jul 26 16:32:19.597312 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.597105    2570 scope.go:117] "RemoveContainer" containerID="ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8"
Jul 26 16:32:19.606311 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.606292    2570 scope.go:117] "RemoveContainer" containerID="ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8"
Jul 26 16:32:19.606556 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:32:19.606537    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8\": container with ID starting with ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8 not found: ID does not exist" containerID="ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8"
Jul 26 16:32:19.606615 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.606567    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8"} err="failed to get container status \"ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8\": rpc error: code = NotFound desc = could not find container \"ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8\": container with ID starting with ee40fd721855cde6bf081022c9a80ce6b6a722bcd46552ea650a84b029e2b6c8 not found: ID does not exist"
Jul 26 16:32:19.618620 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.618594    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"]
Jul 26 16:32:19.621319 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:19.621298    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-aad57859-kserve-b88ff4555-8clqp"]
Jul 26 16:32:20.300080 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:20.300047    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed3656ad-df83-4419-8aef-bdb26af88585" path="/var/lib/kubelet/pods/ed3656ad-df83-4419-8aef-bdb26af88585/volumes"
Jul 26 16:32:20.819233 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:20.819201    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:32:30.737056 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:30.737024    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:32:41.624788 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:41.624754    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:32:51.308297 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:32:51.308258    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:33:00.216202 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:33:00.216155    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:33:08.417852 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:33:08.417817    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:33:19.124632 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:33:19.124597    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:33:24.285568 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:33:24.285540    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:33:24.288158 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:33:24.288137    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:33:28.827468 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:33:28.827434    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:33:41.910271 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:33:41.910235    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:33:49.829463 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:33:49.829375    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:33:59.947780 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:33:59.947743    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:34:10.518910 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:34:10.518866    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:34:21.520447 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:34:21.520413    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:34:28.317039 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:34:28.317003    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:34:43.824885 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:34:43.824851    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:34:55.543095 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:34:55.543061    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:35:07.217406 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:35:07.217364    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:35:16.925669 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:35:16.925584    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:35:38.121989 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:35:38.121956    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:35:51.640271 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:35:51.640234    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:36:27.721207 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:36:27.721154    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:36:39.416877 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:36:39.416843    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:37:16.520441 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:37:16.520405    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:37:28.110218 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:37:28.110168    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:38:00.213022 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:38:00.212981    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:38:11.211680 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:38:11.211648    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:38:24.317384 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:38:24.317351    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:38:24.321808 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:38:24.321789    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:38:45.026862 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:38:45.026825    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:40:11.025958 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:40:11.025881    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:40:22.871435 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:40:22.871391    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:40:35.616123 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:40:35.616088    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:40:53.320153 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:40:53.320119    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:41:30.635167 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:30.635082    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:41:40.535357 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.535326    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"]
Jul 26 16:41:40.535812 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.535679    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3656ad-df83-4419-8aef-bdb26af88585" containerName="main"
Jul 26 16:41:40.535812 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.535690    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3656ad-df83-4419-8aef-bdb26af88585" containerName="main"
Jul 26 16:41:40.535812 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.535751    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3656ad-df83-4419-8aef-bdb26af88585" containerName="main"
Jul 26 16:41:40.538732 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.538715    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.542181 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.542160    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ffa73817\"/\"default-dockercfg-vbb66\""
Jul 26 16:41:40.544086 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.544063    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ffa73817\"/\"e2e-lifecycle-model-ffa73817-kserve-self-signed-certs\""
Jul 26 16:41:40.544216 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.544178    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ffa73817\"/\"openshift-service-ca.crt\""
Jul 26 16:41:40.544216 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.544182    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ffa73817\"/\"kube-root-ca.crt\""
Jul 26 16:41:40.549174 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.549152    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"]
Jul 26 16:41:40.596775 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.596740    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-model-cache\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.596920 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.596789    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tmp-dir\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.596920 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.596842    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-dshm\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.596920 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.596870    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-home\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.597026 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.596921    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tls-certs\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.597026 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.596971    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgrkx\" (UniqueName: \"kubernetes.io/projected/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-kube-api-access-tgrkx\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.697727 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.697693    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tmp-dir\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.697885 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.697741    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-dshm\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.697885 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.697768    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-home\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.697885 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.697798    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tls-certs\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.697885 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.697849    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgrkx\" (UniqueName: \"kubernetes.io/projected/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-kube-api-access-tgrkx\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.698064 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.697911    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-model-cache\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.698167 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.698145    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tmp-dir\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.698268 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.698168    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-home\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.698314 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.698293    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-model-cache\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.700086 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.700065    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-dshm\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.700459 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.700441    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tls-certs\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.706290 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.706273    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgrkx\" (UniqueName: \"kubernetes.io/projected/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-kube-api-access-tgrkx\") pod \"e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") " pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.849248 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.849153    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:40.978775 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:40.978664    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"]
Jul 26 16:41:40.981234 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:41:40.981204    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2279f1aa_84ac_4240_a6e1_8cf60e73bf54.slice/crio-7fcd1bd9fa5625576a3f993c943e12d7048709f0cbdbff1bad4fe1708d889a4b WatchSource:0}: Error finding container 7fcd1bd9fa5625576a3f993c943e12d7048709f0cbdbff1bad4fe1708d889a4b: Status 404 returned error can't find the container with id 7fcd1bd9fa5625576a3f993c943e12d7048709f0cbdbff1bad4fe1708d889a4b
Jul 26 16:41:41.084556 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:41.084522    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp" event={"ID":"2279f1aa-84ac-4240-a6e1-8cf60e73bf54","Type":"ContainerStarted","Data":"e670f546c18f2418453764cef3443148c23e380ec2250c11b56577afb13ec268"}
Jul 26 16:41:41.084692 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:41.084568    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp" event={"ID":"2279f1aa-84ac-4240-a6e1-8cf60e73bf54","Type":"ContainerStarted","Data":"7fcd1bd9fa5625576a3f993c943e12d7048709f0cbdbff1bad4fe1708d889a4b"}
Jul 26 16:41:41.103505 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:41.103104    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp" podStartSLOduration=1.103084332 podStartE2EDuration="1.103084332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:41:41.102917474 +0000 UTC m=+1997.286001762" watchObservedRunningTime="2026-07-26 16:41:41.103084332 +0000 UTC m=+1997.286168624"
Jul 26 16:41:45.619915 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:45.619876    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:41:50.849553 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:50.849505    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:50.850073 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:50.849565    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:50.862136 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:50.862110    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:41:51.142247 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:41:51.142130    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:42:03.524099 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:03.524060    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:42:22.430501 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:22.430464    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:42:30.818999 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:30.818963    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:42:36.122905 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:36.122865    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"]
Jul 26 16:42:51.141593 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.141515    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp" podUID="2279f1aa-84ac-4240-a6e1-8cf60e73bf54" containerName="main" containerID="cri-o://e670f546c18f2418453764cef3443148c23e380ec2250c11b56577afb13ec268" gracePeriod=45
Jul 26 16:42:51.395805 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.395723    2570 generic.go:356] "Generic (PLEG): container finished" podID="2279f1aa-84ac-4240-a6e1-8cf60e73bf54" containerID="e670f546c18f2418453764cef3443148c23e380ec2250c11b56577afb13ec268" exitCode=0
Jul 26 16:42:51.395924 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.395796    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp" event={"ID":"2279f1aa-84ac-4240-a6e1-8cf60e73bf54","Type":"ContainerDied","Data":"e670f546c18f2418453764cef3443148c23e380ec2250c11b56577afb13ec268"}
Jul 26 16:42:51.395924 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.395838    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp" event={"ID":"2279f1aa-84ac-4240-a6e1-8cf60e73bf54","Type":"ContainerDied","Data":"7fcd1bd9fa5625576a3f993c943e12d7048709f0cbdbff1bad4fe1708d889a4b"}
Jul 26 16:42:51.395924 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.395852    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fcd1bd9fa5625576a3f993c943e12d7048709f0cbdbff1bad4fe1708d889a4b"
Jul 26 16:42:51.401666 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.401643    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:42:51.511652 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.511613    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tmp-dir\") pod \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") "
Jul 26 16:42:51.511652 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.511657    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-home\") pod \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") "
Jul 26 16:42:51.511873 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.511700    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-model-cache\") pod \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") "
Jul 26 16:42:51.511873 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.511777    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tls-certs\") pod \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") "
Jul 26 16:42:51.511873 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.511796    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-dshm\") pod \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") "
Jul 26 16:42:51.511873 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.511820    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgrkx\" (UniqueName: \"kubernetes.io/projected/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-kube-api-access-tgrkx\") pod \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\" (UID: \"2279f1aa-84ac-4240-a6e1-8cf60e73bf54\") "
Jul 26 16:42:51.512067 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.511914    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-home" (OuterVolumeSpecName: "home") pod "2279f1aa-84ac-4240-a6e1-8cf60e73bf54" (UID: "2279f1aa-84ac-4240-a6e1-8cf60e73bf54"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:42:51.512067 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.511920    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2279f1aa-84ac-4240-a6e1-8cf60e73bf54" (UID: "2279f1aa-84ac-4240-a6e1-8cf60e73bf54"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:42:51.512067 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.512050    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:42:51.512246 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.512068    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:42:51.512246 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.512111    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-model-cache" (OuterVolumeSpecName: "model-cache") pod "2279f1aa-84ac-4240-a6e1-8cf60e73bf54" (UID: "2279f1aa-84ac-4240-a6e1-8cf60e73bf54"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:42:51.514112 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.514084    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-dshm" (OuterVolumeSpecName: "dshm") pod "2279f1aa-84ac-4240-a6e1-8cf60e73bf54" (UID: "2279f1aa-84ac-4240-a6e1-8cf60e73bf54"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:42:51.514202 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.514155    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-kube-api-access-tgrkx" (OuterVolumeSpecName: "kube-api-access-tgrkx") pod "2279f1aa-84ac-4240-a6e1-8cf60e73bf54" (UID: "2279f1aa-84ac-4240-a6e1-8cf60e73bf54"). InnerVolumeSpecName "kube-api-access-tgrkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:42:51.514253 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.514234    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2279f1aa-84ac-4240-a6e1-8cf60e73bf54" (UID: "2279f1aa-84ac-4240-a6e1-8cf60e73bf54"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:42:51.612593 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.612552    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:42:51.612593 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.612585    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:42:51.612593 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.612595    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgrkx\" (UniqueName: \"kubernetes.io/projected/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-kube-api-access-tgrkx\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:42:51.612818 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:51.612606    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2279f1aa-84ac-4240-a6e1-8cf60e73bf54-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:42:52.400370 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:52.400342    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"
Jul 26 16:42:52.420373 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:52.420340    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"]
Jul 26 16:42:52.423761 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:52.423732    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ffa73817/e2e-lifecycle-model-ffa73817-kserve-6f75b6fc79-tqcgp"]
Jul 26 16:42:54.300960 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:42:54.300927    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2279f1aa-84ac-4240-a6e1-8cf60e73bf54" path="/var/lib/kubelet/pods/2279f1aa-84ac-4240-a6e1-8cf60e73bf54/volumes"
Jul 26 16:43:24.342908 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:43:24.342882    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:43:24.349897 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:43:24.349876    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:43:38.731594 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:43:38.731551    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:43:49.884404 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:43:49.884357    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:44:41.019182 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:44:41.019144    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:44:57.425646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:44:57.425608    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:45:00.140545 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.140512    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l"]
Jul 26 16:45:00.140915 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.140826    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2279f1aa-84ac-4240-a6e1-8cf60e73bf54" containerName="main"
Jul 26 16:45:00.140915 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.140837    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2279f1aa-84ac-4240-a6e1-8cf60e73bf54" containerName="main"
Jul 26 16:45:00.140915 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.140914    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2279f1aa-84ac-4240-a6e1-8cf60e73bf54" containerName="main"
Jul 26 16:45:00.142679 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.142664    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l"
Jul 26 16:45:00.145522 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.145503    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2xlcn\""
Jul 26 16:45:00.152020 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.152000    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l"
Jul 26 16:45:00.158308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.158282    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l"]
Jul 26 16:45:00.185392 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.185361    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"]
Jul 26 16:45:00.188055 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.188034    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.191568 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.191544    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e38070\"/\"kube-root-ca.crt\""
Jul 26 16:45:00.191923 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.191766    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e38070\"/\"test-model-e38070-kserve-self-signed-certs\""
Jul 26 16:45:00.192027 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.191877    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e38070\"/\"default-dockercfg-cqdg8\""
Jul 26 16:45:00.192466 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.192446    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e38070\"/\"openshift-service-ca.crt\""
Jul 26 16:45:00.192543 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.192512    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"]
Jul 26 16:45:00.260069 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.260041    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tmp-dir\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.260250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.260101    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-model-cache\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.260250 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.260235    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-dshm\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.260369 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.260273    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhnvh\" (UniqueName: \"kubernetes.io/projected/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-kube-api-access-hhnvh\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.260369 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.260313    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-home\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.260369 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.260361    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tls-certs\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.289038 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.289005    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l"]
Jul 26 16:45:00.292162 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:45:00.292131    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d3a462_332d_43a2_9928_79dfc049841a.slice/crio-e72d3f2efdb5ebf0f6b8fd58858a721306abf50b51508163fa4972a982035e40 WatchSource:0}: Error finding container e72d3f2efdb5ebf0f6b8fd58858a721306abf50b51508163fa4972a982035e40: Status 404 returned error can't find the container with id e72d3f2efdb5ebf0f6b8fd58858a721306abf50b51508163fa4972a982035e40
Jul 26 16:45:00.294083 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.294065    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:45:00.361134 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.361100    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-dshm\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.361337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.361140    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhnvh\" (UniqueName: \"kubernetes.io/projected/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-kube-api-access-hhnvh\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.361337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.361170    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-home\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.361337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.361234    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tls-certs\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.361337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.361266    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tmp-dir\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.361337 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.361335    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-model-cache\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.361665 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.361641    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-home\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.361785 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.361669    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tmp-dir\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.361785 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.361741    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-model-cache\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.363422 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.363391    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-dshm\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.363711 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.363689    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tls-certs\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.369614 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.369591    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhnvh\" (UniqueName: \"kubernetes.io/projected/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-kube-api-access-hhnvh\") pod \"test-model-e38070-kserve-7599886df5-w9ccr\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") " pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.499997 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.499958    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:00.637510 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.637453    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"]
Jul 26 16:45:00.640078 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:45:00.640054    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d6b56d3_f433_459e_9a8a_16eccb4e7e1d.slice/crio-0c408fc50421685163bcd8107ee0043dab50735f735c83bfe527e77c4a4c33fa WatchSource:0}: Error finding container 0c408fc50421685163bcd8107ee0043dab50735f735c83bfe527e77c4a4c33fa: Status 404 returned error can't find the container with id 0c408fc50421685163bcd8107ee0043dab50735f735c83bfe527e77c4a4c33fa
Jul 26 16:45:00.975559 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.975525    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l" event={"ID":"65d3a462-332d-43a2-9928-79dfc049841a","Type":"ContainerStarted","Data":"e72d3f2efdb5ebf0f6b8fd58858a721306abf50b51508163fa4972a982035e40"}
Jul 26 16:45:00.976890 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.976865    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr" event={"ID":"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d","Type":"ContainerStarted","Data":"d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130"}
Jul 26 16:45:00.976890 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:00.976896    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr" event={"ID":"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d","Type":"ContainerStarted","Data":"0c408fc50421685163bcd8107ee0043dab50735f735c83bfe527e77c4a4c33fa"}
Jul 26 16:45:01.002676 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:01.002626    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr" podStartSLOduration=1.002611714 podStartE2EDuration="1.002611714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:45:01.00121974 +0000 UTC m=+2197.184304027" watchObservedRunningTime="2026-07-26 16:45:01.002611714 +0000 UTC m=+2197.185696002"
Jul 26 16:45:03.993649 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:03.993610    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l" event={"ID":"65d3a462-332d-43a2-9928-79dfc049841a","Type":"ContainerStarted","Data":"13cc88a0e6350e39a2b46e3999af54a6f741540caf7ec3ac6b37b8a3d1f4d7c5"}
Jul 26 16:45:04.009365 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:04.009318    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l" podStartSLOduration=1.337203916 podStartE2EDuration="4.009301353s" totalImagesPullingTime="2.672097437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:45:04.007378264 +0000 UTC m=+2200.190462572" watchObservedRunningTime="2026-07-26 16:45:04.009301353 +0000 UTC m=+2200.192385721"
Jul 26 16:45:04.999005 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:04.998962    2570 generic.go:356] "Generic (PLEG): container finished" podID="65d3a462-332d-43a2-9928-79dfc049841a" containerID="13cc88a0e6350e39a2b46e3999af54a6f741540caf7ec3ac6b37b8a3d1f4d7c5" exitCode=0
Jul 26 16:45:04.999437 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:04.999040    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l" event={"ID":"65d3a462-332d-43a2-9928-79dfc049841a","Type":"ContainerDied","Data":"13cc88a0e6350e39a2b46e3999af54a6f741540caf7ec3ac6b37b8a3d1f4d7c5"}
Jul 26 16:45:06.133927 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:06.133903    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l"
Jul 26 16:45:07.009553 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:07.009519    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l"
Jul 26 16:45:07.009736 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:07.009512    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751405-jnr7l" event={"ID":"65d3a462-332d-43a2-9928-79dfc049841a","Type":"ContainerDied","Data":"e72d3f2efdb5ebf0f6b8fd58858a721306abf50b51508163fa4972a982035e40"}
Jul 26 16:45:07.009736 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:07.009644    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e72d3f2efdb5ebf0f6b8fd58858a721306abf50b51508163fa4972a982035e40"
Jul 26 16:45:10.500715 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:10.500676    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:10.501222 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:10.500738    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:10.513173 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:10.513148    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:11.045794 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:11.045694    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:45:14.114460 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:14.114427    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:45:33.549453 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.549403    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"]
Jul 26 16:45:33.550007 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.549903    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d3a462-332d-43a2-9928-79dfc049841a" containerName="cleanup"
Jul 26 16:45:33.550007 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.549922    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d3a462-332d-43a2-9928-79dfc049841a" containerName="cleanup"
Jul 26 16:45:33.550128 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.550095    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d3a462-332d-43a2-9928-79dfc049841a" containerName="cleanup"
Jul 26 16:45:33.553284 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.553265    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.556240 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.556219    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6a9b62\"/\"openshift-service-ca.crt\""
Jul 26 16:45:33.557667 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.557648    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6a9b62\"/\"kube-root-ca.crt\""
Jul 26 16:45:33.557667 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.557662    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6a9b62\"/\"test-model-6a9b62-kserve-self-signed-certs\""
Jul 26 16:45:33.557805 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.557777    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6a9b62\"/\"default-dockercfg-nwrnv\""
Jul 26 16:45:33.563611 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.563588    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"]
Jul 26 16:45:33.664289 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.664247    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-dshm\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.664289 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.664292    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tmp-dir\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.664521 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.664416    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-home\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.664521 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.664442    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdv7t\" (UniqueName: \"kubernetes.io/projected/fbdc630a-9718-4bf3-96f9-08c76819d1c7-kube-api-access-sdv7t\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.664521 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.664478    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tls-certs\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.664624 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.664569    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-model-cache\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.765144 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.765112    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tls-certs\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.765356 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.765165    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-model-cache\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.765356 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.765216    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-dshm\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.765356 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.765242    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tmp-dir\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.765356 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.765295    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-home\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.765356 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.765318    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdv7t\" (UniqueName: \"kubernetes.io/projected/fbdc630a-9718-4bf3-96f9-08c76819d1c7-kube-api-access-sdv7t\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.765713 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.765678    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-model-cache\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.765713 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.765702    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tmp-dir\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.765833 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.765729    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-home\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.767453 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.767434    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-dshm\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.767619 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.767601    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tls-certs\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.775407 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.775388    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdv7t\" (UniqueName: \"kubernetes.io/projected/fbdc630a-9718-4bf3-96f9-08c76819d1c7-kube-api-access-sdv7t\") pod \"test-model-6a9b62-kserve-547c7f5fb5-8xpfj\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") " pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.863162 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.863068    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:33.996831 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:33.996806    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"]
Jul 26 16:45:33.999105 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:45:33.999079    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbdc630a_9718_4bf3_96f9_08c76819d1c7.slice/crio-b7fa8f133c865cf203e6f6cc792e279ed78a5f273aa30395b2e9d40eb97b4e43 WatchSource:0}: Error finding container b7fa8f133c865cf203e6f6cc792e279ed78a5f273aa30395b2e9d40eb97b4e43: Status 404 returned error can't find the container with id b7fa8f133c865cf203e6f6cc792e279ed78a5f273aa30395b2e9d40eb97b4e43
Jul 26 16:45:34.140241 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:34.140132    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj" event={"ID":"fbdc630a-9718-4bf3-96f9-08c76819d1c7","Type":"ContainerStarted","Data":"b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052"}
Jul 26 16:45:34.140241 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:34.140174    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj" event={"ID":"fbdc630a-9718-4bf3-96f9-08c76819d1c7","Type":"ContainerStarted","Data":"b7fa8f133c865cf203e6f6cc792e279ed78a5f273aa30395b2e9d40eb97b4e43"}
Jul 26 16:45:34.159008 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:34.158962    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj" podStartSLOduration=1.158947294 podStartE2EDuration="1.158947294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:45:34.157821287 +0000 UTC m=+2230.340905575" watchObservedRunningTime="2026-07-26 16:45:34.158947294 +0000 UTC m=+2230.342031580"
Jul 26 16:45:37.716335 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:37.716301    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:45:43.863243 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:43.863182    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:43.863708 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:43.863272    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:43.875941 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:43.875913    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:44.201333 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:44.201304    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:45:51.439440 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:45:51.439346    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:46:19.819689 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:19.819658    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"]
Jul 26 16:46:20.691628 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:20.691592    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"]
Jul 26 16:46:24.822699 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:24.822665    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:46:34.839793 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:34.839734    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr" podUID="1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" containerName="main" containerID="cri-o://d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130" gracePeriod=45
Jul 26 16:46:35.092445 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.092387    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:46:35.118315 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118270    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tmp-dir\") pod \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") "
Jul 26 16:46:35.118477 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118348    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-home\") pod \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") "
Jul 26 16:46:35.118477 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118397    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tls-certs\") pod \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") "
Jul 26 16:46:35.118477 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118452    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-dshm\") pod \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") "
Jul 26 16:46:35.118643 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118490    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-model-cache\") pod \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") "
Jul 26 16:46:35.118643 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118493    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" (UID: "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:46:35.118643 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118544    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhnvh\" (UniqueName: \"kubernetes.io/projected/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-kube-api-access-hhnvh\") pod \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\" (UID: \"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d\") "
Jul 26 16:46:35.118643 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118597    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-home" (OuterVolumeSpecName: "home") pod "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" (UID: "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:46:35.118847 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118777    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-model-cache" (OuterVolumeSpecName: "model-cache") pod "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" (UID: "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:46:35.118916 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118878    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:35.118916 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118895    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:35.118916 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.118910    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:35.120709 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.120676    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-dshm" (OuterVolumeSpecName: "dshm") pod "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" (UID: "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:46:35.120824 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.120730    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" (UID: "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:46:35.120824 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.120747    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-kube-api-access-hhnvh" (OuterVolumeSpecName: "kube-api-access-hhnvh") pod "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" (UID: "1d6b56d3-f433-459e-9a8a-16eccb4e7e1d"). InnerVolumeSpecName "kube-api-access-hhnvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:46:35.220115 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.220075    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:35.220115 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.220102    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:35.220115 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.220111    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhnvh\" (UniqueName: \"kubernetes.io/projected/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d-kube-api-access-hhnvh\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:35.424359 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.424276    2570 generic.go:356] "Generic (PLEG): container finished" podID="1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" containerID="d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130" exitCode=0
Jul 26 16:46:35.424359 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.424319    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"
Jul 26 16:46:35.424537 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.424362    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr" event={"ID":"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d","Type":"ContainerDied","Data":"d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130"}
Jul 26 16:46:35.424537 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.424402    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr" event={"ID":"1d6b56d3-f433-459e-9a8a-16eccb4e7e1d","Type":"ContainerDied","Data":"0c408fc50421685163bcd8107ee0043dab50735f735c83bfe527e77c4a4c33fa"}
Jul 26 16:46:35.424537 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.424420    2570 scope.go:117] "RemoveContainer" containerID="d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130"
Jul 26 16:46:35.433389 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.433369    2570 scope.go:117] "RemoveContainer" containerID="d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130"
Jul 26 16:46:35.433652 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:46:35.433628    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130\": container with ID starting with d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130 not found: ID does not exist" containerID="d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130"
Jul 26 16:46:35.433717 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.433661    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130"} err="failed to get container status \"d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130\": rpc error: code = NotFound desc = could not find container \"d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130\": container with ID starting with d707bca96aae376defd57451026c804131b1f30fe2a6dd85defdc9bf285ba130 not found: ID does not exist"
Jul 26 16:46:35.445635 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.445608    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"]
Jul 26 16:46:35.449788 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.449768    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e38070/test-model-e38070-kserve-7599886df5-w9ccr"]
Jul 26 16:46:35.710824 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.710780    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj" podUID="fbdc630a-9718-4bf3-96f9-08c76819d1c7" containerName="main" containerID="cri-o://b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052" gracePeriod=45
Jul 26 16:46:35.953280 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:35.953252    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:46:36.026514 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.026413    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-dshm\") pod \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") "
Jul 26 16:46:36.026514 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.026460    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdv7t\" (UniqueName: \"kubernetes.io/projected/fbdc630a-9718-4bf3-96f9-08c76819d1c7-kube-api-access-sdv7t\") pod \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") "
Jul 26 16:46:36.026745 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.026524    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tls-certs\") pod \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") "
Jul 26 16:46:36.026745 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.026556    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tmp-dir\") pod \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") "
Jul 26 16:46:36.026745 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.026580    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-model-cache\") pod \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") "
Jul 26 16:46:36.026745 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.026631    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-home\") pod \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\" (UID: \"fbdc630a-9718-4bf3-96f9-08c76819d1c7\") "
Jul 26 16:46:36.026963 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.026898    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fbdc630a-9718-4bf3-96f9-08c76819d1c7" (UID: "fbdc630a-9718-4bf3-96f9-08c76819d1c7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:46:36.027062 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.027033    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-model-cache" (OuterVolumeSpecName: "model-cache") pod "fbdc630a-9718-4bf3-96f9-08c76819d1c7" (UID: "fbdc630a-9718-4bf3-96f9-08c76819d1c7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:46:36.027139 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.027059    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-home" (OuterVolumeSpecName: "home") pod "fbdc630a-9718-4bf3-96f9-08c76819d1c7" (UID: "fbdc630a-9718-4bf3-96f9-08c76819d1c7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:46:36.029264 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.029236    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fbdc630a-9718-4bf3-96f9-08c76819d1c7" (UID: "fbdc630a-9718-4bf3-96f9-08c76819d1c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:46:36.029371 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.029236    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-dshm" (OuterVolumeSpecName: "dshm") pod "fbdc630a-9718-4bf3-96f9-08c76819d1c7" (UID: "fbdc630a-9718-4bf3-96f9-08c76819d1c7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:46:36.029449 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.029432    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbdc630a-9718-4bf3-96f9-08c76819d1c7-kube-api-access-sdv7t" (OuterVolumeSpecName: "kube-api-access-sdv7t") pod "fbdc630a-9718-4bf3-96f9-08c76819d1c7" (UID: "fbdc630a-9718-4bf3-96f9-08c76819d1c7"). InnerVolumeSpecName "kube-api-access-sdv7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:46:36.128115 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.128085    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:36.128115 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.128111    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:36.128115 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.128120    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:36.128344 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.128128    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:36.128344 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.128136    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbdc630a-9718-4bf3-96f9-08c76819d1c7-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:36.128344 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.128148    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdv7t\" (UniqueName: \"kubernetes.io/projected/fbdc630a-9718-4bf3-96f9-08c76819d1c7-kube-api-access-sdv7t\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:36.300125 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.300036    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" path="/var/lib/kubelet/pods/1d6b56d3-f433-459e-9a8a-16eccb4e7e1d/volumes"
Jul 26 16:46:36.436850 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.436810    2570 generic.go:356] "Generic (PLEG): container finished" podID="fbdc630a-9718-4bf3-96f9-08c76819d1c7" containerID="b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052" exitCode=0
Jul 26 16:46:36.437018 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.436861    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj" event={"ID":"fbdc630a-9718-4bf3-96f9-08c76819d1c7","Type":"ContainerDied","Data":"b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052"}
Jul 26 16:46:36.437018 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.436868    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"
Jul 26 16:46:36.437018 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.436902    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj" event={"ID":"fbdc630a-9718-4bf3-96f9-08c76819d1c7","Type":"ContainerDied","Data":"b7fa8f133c865cf203e6f6cc792e279ed78a5f273aa30395b2e9d40eb97b4e43"}
Jul 26 16:46:36.437018 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.436923    2570 scope.go:117] "RemoveContainer" containerID="b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052"
Jul 26 16:46:36.445398 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.445377    2570 scope.go:117] "RemoveContainer" containerID="b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052"
Jul 26 16:46:36.445657 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:46:36.445637    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052\": container with ID starting with b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052 not found: ID does not exist" containerID="b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052"
Jul 26 16:46:36.445734 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.445668    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052"} err="failed to get container status \"b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052\": rpc error: code = NotFound desc = could not find container \"b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052\": container with ID starting with b471cb7790eb194e794a6fbd4493bd2552eace83aaf1f72d0f46b549c716e052 not found: ID does not exist"
Jul 26 16:46:36.457111 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.457084    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"]
Jul 26 16:46:36.459245 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:36.459222    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-6a9b62/test-model-6a9b62-kserve-547c7f5fb5-8xpfj"]
Jul 26 16:46:37.216050 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:37.216014    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:46:38.300399 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:38.300365    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbdc630a-9718-4bf3-96f9-08c76819d1c7" path="/var/lib/kubelet/pods/fbdc630a-9718-4bf3-96f9-08c76819d1c7/volumes"
Jul 26 16:46:42.633609 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.633578    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"]
Jul 26 16:46:42.634008 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.633929    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" containerName="main"
Jul 26 16:46:42.634008 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.633939    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" containerName="main"
Jul 26 16:46:42.634008 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.633958    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbdc630a-9718-4bf3-96f9-08c76819d1c7" containerName="main"
Jul 26 16:46:42.634008 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.633963    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbdc630a-9718-4bf3-96f9-08c76819d1c7" containerName="main"
Jul 26 16:46:42.634008 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.634006    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d6b56d3-f433-459e-9a8a-16eccb4e7e1d" containerName="main"
Jul 26 16:46:42.634180 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.634019    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbdc630a-9718-4bf3-96f9-08c76819d1c7" containerName="main"
Jul 26 16:46:42.638361 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.638344    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"
Jul 26 16:46:42.641397 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.641376    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2xlcn\""
Jul 26 16:46:42.641527 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.641433    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 16:46:42.645506 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.645242    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"]
Jul 26 16:46:42.686948 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.686911    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b96323-4572-42b2-be43-d7bd6778b37e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc\" (UID: \"72b96323-4572-42b2-be43-d7bd6778b37e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"
Jul 26 16:46:42.788259 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.788224    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b96323-4572-42b2-be43-d7bd6778b37e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc\" (UID: \"72b96323-4572-42b2-be43-d7bd6778b37e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"
Jul 26 16:46:42.788960 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.788937    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b96323-4572-42b2-be43-d7bd6778b37e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc\" (UID: \"72b96323-4572-42b2-be43-d7bd6778b37e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"
Jul 26 16:46:42.949800 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:42.949768    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"
Jul 26 16:46:43.079962 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:43.079937    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"]
Jul 26 16:46:43.082444 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:46:43.082411    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b96323_4572_42b2_be43_d7bd6778b37e.slice/crio-d69689ce12cd791dfa69908cfa7f254fd4c75e0f50f5229afdbc572e96e8fad1 WatchSource:0}: Error finding container d69689ce12cd791dfa69908cfa7f254fd4c75e0f50f5229afdbc572e96e8fad1: Status 404 returned error can't find the container with id d69689ce12cd791dfa69908cfa7f254fd4c75e0f50f5229afdbc572e96e8fad1
Jul 26 16:46:43.471792 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:43.471752    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc" event={"ID":"72b96323-4572-42b2-be43-d7bd6778b37e","Type":"ContainerStarted","Data":"82fa9a6ee29d8a65e3be8fe387ef9f0c1a9c15d6e6817175c61bb880cc37ba65"}
Jul 26 16:46:43.471927 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:43.471808    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc" event={"ID":"72b96323-4572-42b2-be43-d7bd6778b37e","Type":"ContainerStarted","Data":"d69689ce12cd791dfa69908cfa7f254fd4c75e0f50f5229afdbc572e96e8fad1"}
Jul 26 16:46:43.491684 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:43.491626    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc" podStartSLOduration=1.491607689 podStartE2EDuration="1.491607689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:46:43.489753256 +0000 UTC m=+2299.672837566" watchObservedRunningTime="2026-07-26 16:46:43.491607689 +0000 UTC m=+2299.674691976"
Jul 26 16:46:44.478037 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:44.478000    2570 generic.go:356] "Generic (PLEG): container finished" podID="72b96323-4572-42b2-be43-d7bd6778b37e" containerID="82fa9a6ee29d8a65e3be8fe387ef9f0c1a9c15d6e6817175c61bb880cc37ba65" exitCode=0
Jul 26 16:46:44.478407 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:44.478066    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc" event={"ID":"72b96323-4572-42b2-be43-d7bd6778b37e","Type":"ContainerDied","Data":"82fa9a6ee29d8a65e3be8fe387ef9f0c1a9c15d6e6817175c61bb880cc37ba65"}
Jul 26 16:46:45.607275 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:45.607253    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"
Jul 26 16:46:45.712580 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:45.712548    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b96323-4572-42b2-be43-d7bd6778b37e-maas-api-service-ca\") pod \"72b96323-4572-42b2-be43-d7bd6778b37e\" (UID: \"72b96323-4572-42b2-be43-d7bd6778b37e\") "
Jul 26 16:46:45.712908 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:45.712883    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72b96323-4572-42b2-be43-d7bd6778b37e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72b96323-4572-42b2-be43-d7bd6778b37e" (UID: "72b96323-4572-42b2-be43-d7bd6778b37e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:46:45.813917 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:45.813837    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b96323-4572-42b2-be43-d7bd6778b37e-maas-api-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:46:46.488354 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:46.488321    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"
Jul 26 16:46:46.488512 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:46.488320    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc" event={"ID":"72b96323-4572-42b2-be43-d7bd6778b37e","Type":"ContainerDied","Data":"d69689ce12cd791dfa69908cfa7f254fd4c75e0f50f5229afdbc572e96e8fad1"}
Jul 26 16:46:46.488512 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:46.488459    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d69689ce12cd791dfa69908cfa7f254fd4c75e0f50f5229afdbc572e96e8fad1"
Jul 26 16:46:55.815458 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:55.815420    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:46:57.681719 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:57.681683    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"]
Jul 26 16:46:57.684606 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:57.684582    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6a9b62-s2jcc"]
Jul 26 16:46:58.300022 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:46:58.299988    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72b96323-4572-42b2-be43-d7bd6778b37e" path="/var/lib/kubelet/pods/72b96323-4572-42b2-be43-d7bd6778b37e/volumes"
Jul 26 16:47:14.922839 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:14.922802    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:47:28.425880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.425822    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"]
Jul 26 16:47:28.426583 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.426561    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72b96323-4572-42b2-be43-d7bd6778b37e" containerName="revoke-keys"
Jul 26 16:47:28.426654 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.426587    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="72b96323-4572-42b2-be43-d7bd6778b37e" containerName="revoke-keys"
Jul 26 16:47:28.426775 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.426762    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="72b96323-4572-42b2-be43-d7bd6778b37e" containerName="revoke-keys"
Jul 26 16:47:28.432112 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.432085    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"
Jul 26 16:47:28.434405 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.434378    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"]
Jul 26 16:47:28.435353 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.435332    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 16:47:28.435458 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.435378    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2xlcn\""
Jul 26 16:47:28.504924 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.504888    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/585b859a-bc10-4c53-9bef-aaee43e93757-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f\" (UID: \"585b859a-bc10-4c53-9bef-aaee43e93757\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"
Jul 26 16:47:28.606060 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.606029    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/585b859a-bc10-4c53-9bef-aaee43e93757-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f\" (UID: \"585b859a-bc10-4c53-9bef-aaee43e93757\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"
Jul 26 16:47:28.606646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.606626    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/585b859a-bc10-4c53-9bef-aaee43e93757-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f\" (UID: \"585b859a-bc10-4c53-9bef-aaee43e93757\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"
Jul 26 16:47:28.743320 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.743287    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"
Jul 26 16:47:28.876402 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:28.876380    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"]
Jul 26 16:47:28.878156 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:47:28.878125    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585b859a_bc10_4c53_9bef_aaee43e93757.slice/crio-83dabdd20d042df11b6157ed72ed811d21c625dc4f9d6d54e951afea73dcb422 WatchSource:0}: Error finding container 83dabdd20d042df11b6157ed72ed811d21c625dc4f9d6d54e951afea73dcb422: Status 404 returned error can't find the container with id 83dabdd20d042df11b6157ed72ed811d21c625dc4f9d6d54e951afea73dcb422
Jul 26 16:47:29.690644 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:29.690606    2570 generic.go:356] "Generic (PLEG): container finished" podID="585b859a-bc10-4c53-9bef-aaee43e93757" containerID="c495c1e73cb884ae707017c05127aad5eb3ebbbd19034258ed130ad79d3cc99f" exitCode=0
Jul 26 16:47:29.691039 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:29.690695    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f" event={"ID":"585b859a-bc10-4c53-9bef-aaee43e93757","Type":"ContainerDied","Data":"c495c1e73cb884ae707017c05127aad5eb3ebbbd19034258ed130ad79d3cc99f"}
Jul 26 16:47:29.691039 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:29.690746    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f" event={"ID":"585b859a-bc10-4c53-9bef-aaee43e93757","Type":"ContainerStarted","Data":"83dabdd20d042df11b6157ed72ed811d21c625dc4f9d6d54e951afea73dcb422"}
Jul 26 16:47:30.820335 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:30.820309    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"
Jul 26 16:47:30.927044 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:30.927006    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/585b859a-bc10-4c53-9bef-aaee43e93757-maas-api-service-ca\") pod \"585b859a-bc10-4c53-9bef-aaee43e93757\" (UID: \"585b859a-bc10-4c53-9bef-aaee43e93757\") "
Jul 26 16:47:30.927382 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:30.927360    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/585b859a-bc10-4c53-9bef-aaee43e93757-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "585b859a-bc10-4c53-9bef-aaee43e93757" (UID: "585b859a-bc10-4c53-9bef-aaee43e93757"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:47:31.028236 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:31.028124    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/585b859a-bc10-4c53-9bef-aaee43e93757-maas-api-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:47:31.701244 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:31.701210    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f" event={"ID":"585b859a-bc10-4c53-9bef-aaee43e93757","Type":"ContainerDied","Data":"83dabdd20d042df11b6157ed72ed811d21c625dc4f9d6d54e951afea73dcb422"}
Jul 26 16:47:31.701244 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:31.701232    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"
Jul 26 16:47:31.701442 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:31.701262    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83dabdd20d042df11b6157ed72ed811d21c625dc4f9d6d54e951afea73dcb422"
Jul 26 16:47:32.847768 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:32.847733    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:47:48.537291 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:48.537253    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"]
Jul 26 16:47:48.542386 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:48.542357    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5425d5-8ll2f"]
Jul 26 16:47:48.929466 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:48.929375    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:47:50.299447 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:50.299413    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="585b859a-bc10-4c53-9bef-aaee43e93757" path="/var/lib/kubelet/pods/585b859a-bc10-4c53-9bef-aaee43e93757/volumes"
Jul 26 16:47:55.367994 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.367948    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"]
Jul 26 16:47:55.368527 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.368506    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585b859a-bc10-4c53-9bef-aaee43e93757" containerName="revoke-keys"
Jul 26 16:47:55.368601 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.368529    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="585b859a-bc10-4c53-9bef-aaee43e93757" containerName="revoke-keys"
Jul 26 16:47:55.368654 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.368627    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="585b859a-bc10-4c53-9bef-aaee43e93757" containerName="revoke-keys"
Jul 26 16:47:55.371649 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.371629    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"
Jul 26 16:47:55.374484 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.374460    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2xlcn\""
Jul 26 16:47:55.374592 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.374490    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 16:47:55.379810 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.379788    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"]
Jul 26 16:47:55.438674 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.438632    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k\" (UID: \"0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"
Jul 26 16:47:55.539399 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.539368    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k\" (UID: \"0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"
Jul 26 16:47:55.539993 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.539973    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k\" (UID: \"0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"
Jul 26 16:47:55.683450 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.683369    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"
Jul 26 16:47:55.827336 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:55.827311    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"]
Jul 26 16:47:55.829237 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:47:55.829212    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b23b6ea_a7c7_494f_b1a2_4711a6e39f5f.slice/crio-1be117691b3e0fc5a6424863e53f4b626c974f16540db6f4e6c037951dd1fa44 WatchSource:0}: Error finding container 1be117691b3e0fc5a6424863e53f4b626c974f16540db6f4e6c037951dd1fa44: Status 404 returned error can't find the container with id 1be117691b3e0fc5a6424863e53f4b626c974f16540db6f4e6c037951dd1fa44
Jul 26 16:47:56.822585 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:56.822493    2570 generic.go:356] "Generic (PLEG): container finished" podID="0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f" containerID="f0c94f9363b4306688a117e1dcb148f76704dbd50503080bcd969a76461fbb0c" exitCode=0
Jul 26 16:47:56.822585 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:56.822574    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k" event={"ID":"0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f","Type":"ContainerDied","Data":"f0c94f9363b4306688a117e1dcb148f76704dbd50503080bcd969a76461fbb0c"}
Jul 26 16:47:56.822998 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:56.822647    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k" event={"ID":"0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f","Type":"ContainerStarted","Data":"1be117691b3e0fc5a6424863e53f4b626c974f16540db6f4e6c037951dd1fa44"}
Jul 26 16:47:57.963493 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:57.963465    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"
Jul 26 16:47:58.059891 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:58.059857    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f-maas-api-service-ca\") pod \"0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f\" (UID: \"0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f\") "
Jul 26 16:47:58.060251 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:58.060227    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f" (UID: "0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:47:58.160992 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:58.160912    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f-maas-api-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:47:58.832363 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:58.832336    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"
Jul 26 16:47:58.832550 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:58.832336    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k" event={"ID":"0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f","Type":"ContainerDied","Data":"1be117691b3e0fc5a6424863e53f4b626c974f16540db6f4e6c037951dd1fa44"}
Jul 26 16:47:58.832550 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:47:58.832456    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1be117691b3e0fc5a6424863e53f4b626c974f16540db6f4e6c037951dd1fa44"
Jul 26 16:48:15.459362 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:15.459326    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"]
Jul 26 16:48:15.462272 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:15.462247    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ddd8f8-k9p8k"]
Jul 26 16:48:15.922538 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:15.922453    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:48:16.300449 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:16.300421    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f" path="/var/lib/kubelet/pods/0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f/volumes"
Jul 26 16:48:24.379880 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:24.379846    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:48:24.386301 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:24.386280    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:48:24.424960 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:24.424935    2570 scope.go:117] "RemoveContainer" containerID="e670f546c18f2418453764cef3443148c23e380ec2250c11b56577afb13ec268"
Jul 26 16:48:35.140163 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:35.140125    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:48:49.355592 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:49.355505    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:48:55.463254 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.463200    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"]
Jul 26 16:48:55.463705 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.463536    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f" containerName="revoke-keys"
Jul 26 16:48:55.463705 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.463548    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f" containerName="revoke-keys"
Jul 26 16:48:55.463705 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.463620    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b23b6ea-a7c7-494f-b1a2-4711a6e39f5f" containerName="revoke-keys"
Jul 26 16:48:55.466536 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.466517    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.471452 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.471416    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"kube-root-ca.crt\""
Jul 26 16:48:55.471594 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.471454    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"auth-test-model-1b2072-kserve-self-signed-certs\""
Jul 26 16:48:55.471594 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.471467    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"openshift-service-ca.crt\""
Jul 26 16:48:55.471594 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.471459    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"default-dockercfg-5r7cf\""
Jul 26 16:48:55.481100 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.481077    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"]
Jul 26 16:48:55.600990 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.600945    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-dshm\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.601160 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.600989    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tls-certs\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.601160 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.601069    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tmp-dir\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.601160 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.601110    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hg94\" (UniqueName: \"kubernetes.io/projected/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-kube-api-access-4hg94\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.601288 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.601227    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-model-cache\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.601288 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.601248    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-home\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.702021 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.701979    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-dshm\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.702021 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.702017    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tls-certs\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.702246 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.702038    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tmp-dir\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.702246 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.702166    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hg94\" (UniqueName: \"kubernetes.io/projected/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-kube-api-access-4hg94\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.702326 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.702308    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-model-cache\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.702364 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.702343    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-home\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.702523 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.702497    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tmp-dir\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.702694 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.702668    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-model-cache\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.702694 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.702689    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-home\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.704308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.704284    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-dshm\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.704597 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.704575    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tls-certs\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.710714 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.710691    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hg94\" (UniqueName: \"kubernetes.io/projected/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-kube-api-access-4hg94\") pod \"auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") " pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.775869 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.775784    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:48:55.912799 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:55.912771    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"]
Jul 26 16:48:55.915164 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:48:55.915136    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9fe8ffa_957f_4268_b3ef_d6405fd244c5.slice/crio-a5607f608a5a34470aa011f2eb479437d8ef41b4a9419080ab2a8816a2396a42 WatchSource:0}: Error finding container a5607f608a5a34470aa011f2eb479437d8ef41b4a9419080ab2a8816a2396a42: Status 404 returned error can't find the container with id a5607f608a5a34470aa011f2eb479437d8ef41b4a9419080ab2a8816a2396a42
Jul 26 16:48:56.096265 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:56.096165    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh" event={"ID":"e9fe8ffa-957f-4268-b3ef-d6405fd244c5","Type":"ContainerStarted","Data":"0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03"}
Jul 26 16:48:56.096265 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:56.096222    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh" event={"ID":"e9fe8ffa-957f-4268-b3ef-d6405fd244c5","Type":"ContainerStarted","Data":"a5607f608a5a34470aa011f2eb479437d8ef41b4a9419080ab2a8816a2396a42"}
Jul 26 16:48:56.118822 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:48:56.118755    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh" podStartSLOduration=1.11873179 podStartE2EDuration="1.11873179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:48:56.115859957 +0000 UTC m=+2432.298944245" watchObservedRunningTime="2026-07-26 16:48:56.11873179 +0000 UTC m=+2432.301816079"
Jul 26 16:49:00.624589 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:49:00.624553    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:49:05.776708 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:49:05.776671    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:49:05.777096 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:49:05.776725    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:49:05.789111 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:49:05.789087    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:49:06.151986 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:49:06.151901    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:49:43.579712 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:49:43.579674    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:50:11.421105 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:50:11.421067    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:50:39.127276 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:50:39.127230    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:50:53.044332 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:50:53.044297    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:51:04.771017 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:51:04.770978    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:51:24.371031 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:51:24.370996    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:51:43.563642 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:51:43.563603    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:52:03.840653 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:52:03.840569    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:52:22.430732 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:52:22.430695    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:52:43.849678 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:52:43.849640    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:52:57.026162 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:52:57.026128    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:53:09.850647 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:53:09.850612    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:53:24.412855 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:53:24.412782    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:53:24.415943 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:53:24.415924    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:53:24.471359 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:53:24.471333    2570 scope.go:117] "RemoveContainer" containerID="82fa9a6ee29d8a65e3be8fe387ef9f0c1a9c15d6e6817175c61bb880cc37ba65"
Jul 26 16:53:27.540096 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:53:27.540062    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:53:45.628861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:53:45.628824    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:54:07.128866 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:07.128828    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:54:10.048874 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:10.048839    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"]
Jul 26 16:54:23.619866 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:23.619832    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:54:24.489533 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:24.489500    2570 scope.go:117] "RemoveContainer" containerID="f0c94f9363b4306688a117e1dcb148f76704dbd50503080bcd969a76461fbb0c"
Jul 26 16:54:24.497499 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:24.497482    2570 scope.go:117] "RemoveContainer" containerID="c495c1e73cb884ae707017c05127aad5eb3ebbbd19034258ed130ad79d3cc99f"
Jul 26 16:54:25.073258 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.073184    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh" podUID="e9fe8ffa-957f-4268-b3ef-d6405fd244c5" containerName="main" containerID="cri-o://0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03" gracePeriod=45
Jul 26 16:54:25.329004 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.328944    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:54:25.477548 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.477510    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hg94\" (UniqueName: \"kubernetes.io/projected/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-kube-api-access-4hg94\") pod \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") "
Jul 26 16:54:25.477714 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.477581    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tmp-dir\") pod \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") "
Jul 26 16:54:25.477714 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.477608    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tls-certs\") pod \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") "
Jul 26 16:54:25.477714 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.477655    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-home\") pod \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") "
Jul 26 16:54:25.477714 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.477682    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-model-cache\") pod \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") "
Jul 26 16:54:25.477875 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.477733    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-dshm\") pod \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\" (UID: \"e9fe8ffa-957f-4268-b3ef-d6405fd244c5\") "
Jul 26 16:54:25.477931 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.477904    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9fe8ffa-957f-4268-b3ef-d6405fd244c5" (UID: "e9fe8ffa-957f-4268-b3ef-d6405fd244c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:54:25.477990 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.477952    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-home" (OuterVolumeSpecName: "home") pod "e9fe8ffa-957f-4268-b3ef-d6405fd244c5" (UID: "e9fe8ffa-957f-4268-b3ef-d6405fd244c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:54:25.478048 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.477975    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9fe8ffa-957f-4268-b3ef-d6405fd244c5" (UID: "e9fe8ffa-957f-4268-b3ef-d6405fd244c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:54:25.478048 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.478032    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:54:25.479860 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.479829    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9fe8ffa-957f-4268-b3ef-d6405fd244c5" (UID: "e9fe8ffa-957f-4268-b3ef-d6405fd244c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:54:25.479980 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.479865    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-dshm" (OuterVolumeSpecName: "dshm") pod "e9fe8ffa-957f-4268-b3ef-d6405fd244c5" (UID: "e9fe8ffa-957f-4268-b3ef-d6405fd244c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:54:25.479980 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.479933    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-kube-api-access-4hg94" (OuterVolumeSpecName: "kube-api-access-4hg94") pod "e9fe8ffa-957f-4268-b3ef-d6405fd244c5" (UID: "e9fe8ffa-957f-4268-b3ef-d6405fd244c5"). InnerVolumeSpecName "kube-api-access-4hg94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:54:25.579173 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.579099    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:54:25.579173 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.579126    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:54:25.579173 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.579136    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:54:25.579173 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.579144    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hg94\" (UniqueName: \"kubernetes.io/projected/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-kube-api-access-4hg94\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:54:25.579173 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.579153    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9fe8ffa-957f-4268-b3ef-d6405fd244c5-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:54:25.595822 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.595786    2570 generic.go:356] "Generic (PLEG): container finished" podID="e9fe8ffa-957f-4268-b3ef-d6405fd244c5" containerID="0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03" exitCode=0
Jul 26 16:54:25.595940 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.595833    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"
Jul 26 16:54:25.595940 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.595873    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh" event={"ID":"e9fe8ffa-957f-4268-b3ef-d6405fd244c5","Type":"ContainerDied","Data":"0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03"}
Jul 26 16:54:25.595940 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.595918    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh" event={"ID":"e9fe8ffa-957f-4268-b3ef-d6405fd244c5","Type":"ContainerDied","Data":"a5607f608a5a34470aa011f2eb479437d8ef41b4a9419080ab2a8816a2396a42"}
Jul 26 16:54:25.595940 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.595936    2570 scope.go:117] "RemoveContainer" containerID="0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03"
Jul 26 16:54:25.604400 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.604385    2570 scope.go:117] "RemoveContainer" containerID="0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03"
Jul 26 16:54:25.604632 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:54:25.604616    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03\": container with ID starting with 0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03 not found: ID does not exist" containerID="0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03"
Jul 26 16:54:25.604674 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.604639    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03"} err="failed to get container status \"0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03\": rpc error: code = NotFound desc = could not find container \"0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03\": container with ID starting with 0842ea95fc0f1eb3cdcbcebc80e511bec9910d7fc791f54e14fe6ddba8da4b03 not found: ID does not exist"
Jul 26 16:54:25.619569 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.619545    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"]
Jul 26 16:54:25.622904 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:25.622885    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/auth-test-model-1b2072-kserve-6f486fb6cf-zb5vh"]
Jul 26 16:54:26.299725 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:26.299694    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9fe8ffa-957f-4268-b3ef-d6405fd244c5" path="/var/lib/kubelet/pods/e9fe8ffa-957f-4268-b3ef-d6405fd244c5/volumes"
Jul 26 16:54:40.862666 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.862623    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"]
Jul 26 16:54:40.863132 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.862951    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9fe8ffa-957f-4268-b3ef-d6405fd244c5" containerName="main"
Jul 26 16:54:40.863132 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.862961    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9fe8ffa-957f-4268-b3ef-d6405fd244c5" containerName="main"
Jul 26 16:54:40.863132 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.863041    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9fe8ffa-957f-4268-b3ef-d6405fd244c5" containerName="main"
Jul 26 16:54:40.865967 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.865951    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:40.869488 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.869467    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"kube-root-ca.crt\""
Jul 26 16:54:40.870303 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.870285    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"default-dockercfg-5r7cf\""
Jul 26 16:54:40.871081 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.871062    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"sub-test-model-1b2072-kserve-self-signed-certs\""
Jul 26 16:54:40.871208 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.871120    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"openshift-service-ca.crt\""
Jul 26 16:54:40.877738 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.877714    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"]
Jul 26 16:54:40.911503 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.911473    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tmp-dir\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:40.911646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.911517    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-dshm\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:40.911646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.911579    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-model-cache\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:40.911646 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.911642    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-home\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:40.911799 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.911670    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tls-certs\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:40.911799 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:40.911699    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh2tz\" (UniqueName: \"kubernetes.io/projected/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-kube-api-access-xh2tz\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.012442 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.012408    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-dshm\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.012442 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.012454    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-model-cache\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.012984 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.012494    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-home\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.012984 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.012520    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tls-certs\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.012984 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.012543    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh2tz\" (UniqueName: \"kubernetes.io/projected/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-kube-api-access-xh2tz\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.012984 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.012606    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tmp-dir\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.012984 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.012952    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-home\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.012984 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.012963    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-model-cache\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.013359 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.013019    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tmp-dir\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.014801 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.014775    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-dshm\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.015553 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.015532    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tls-certs\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.022236 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.022209    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh2tz\" (UniqueName: \"kubernetes.io/projected/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-kube-api-access-xh2tz\") pod \"sub-test-model-1b2072-kserve-7c7c845975-hhh2b\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") " pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.175288 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.175204    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:41.309999 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.309973    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"]
Jul 26 16:54:41.312097 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:54:41.312067    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda95c7946_fcd8_4b1d_a579_65632ef3b1d6.slice/crio-8770049e9bff101c54aa4e37ec7a96a780f2e7d08298c5c6bf812743941e2b60 WatchSource:0}: Error finding container 8770049e9bff101c54aa4e37ec7a96a780f2e7d08298c5c6bf812743941e2b60: Status 404 returned error can't find the container with id 8770049e9bff101c54aa4e37ec7a96a780f2e7d08298c5c6bf812743941e2b60
Jul 26 16:54:41.668424 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.668389    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b" event={"ID":"a95c7946-fcd8-4b1d-a579-65632ef3b1d6","Type":"ContainerStarted","Data":"bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3"}
Jul 26 16:54:41.668584 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.668431    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b" event={"ID":"a95c7946-fcd8-4b1d-a579-65632ef3b1d6","Type":"ContainerStarted","Data":"8770049e9bff101c54aa4e37ec7a96a780f2e7d08298c5c6bf812743941e2b60"}
Jul 26 16:54:41.693805 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:41.693755    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b" podStartSLOduration=1.693740022 podStartE2EDuration="1.693740022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:54:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:54:41.692116593 +0000 UTC m=+2777.875200879" watchObservedRunningTime="2026-07-26 16:54:41.693740022 +0000 UTC m=+2777.876824314"
Jul 26 16:54:44.822311 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:44.822277    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:54:51.176016 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:51.175925    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:51.176016 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:51.175995    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:51.188847 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:51.188821    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:54:51.724628 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:54:51.724601    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:55:36.576038 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.575995    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"]
Jul 26 16:55:36.579109 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.579091    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.582382 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.582364    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a06722\"/\"kube-root-ca.crt\""
Jul 26 16:55:36.582483 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.582384    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a06722\"/\"openshift-service-ca.crt\""
Jul 26 16:55:36.583770 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.583751    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a06722\"/\"default-dockercfg-4bmgt\""
Jul 26 16:55:36.583857 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.583792    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a06722\"/\"sub-test-model-a06722-kserve-self-signed-certs\""
Jul 26 16:55:36.592574 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.592550    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"]
Jul 26 16:55:36.707170 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.707139    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-model-cache\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.707357 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.707177    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tmp-dir\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.707357 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.707212    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzrgz\" (UniqueName: \"kubernetes.io/projected/019bdf09-84fb-48ff-9ca8-44bbe17ca127-kube-api-access-fzrgz\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.707357 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.707323    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-dshm\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.707518 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.707367    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tls-certs\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.707518 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.707426    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-home\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.808945 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.808901    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-model-cache\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.808945 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.808947    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tmp-dir\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.809225 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.808965    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzrgz\" (UniqueName: \"kubernetes.io/projected/019bdf09-84fb-48ff-9ca8-44bbe17ca127-kube-api-access-fzrgz\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.809225 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.809013    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-dshm\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.809225 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.809032    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tls-certs\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.809225 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.809073    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-home\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.809411 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.809388    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tmp-dir\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.809477 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.809439    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-model-cache\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.809547 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.809530    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-home\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.811395 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.811375    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-dshm\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.811513 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.811462    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tls-certs\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.820690 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.820661    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzrgz\" (UniqueName: \"kubernetes.io/projected/019bdf09-84fb-48ff-9ca8-44bbe17ca127-kube-api-access-fzrgz\") pod \"sub-test-model-a06722-kserve-55bc45655f-r5rmh\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") " pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:36.888393 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:36.888301    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:37.023829 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:37.023794    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"]
Jul 26 16:55:37.026540 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:55:37.026509    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod019bdf09_84fb_48ff_9ca8_44bbe17ca127.slice/crio-efeb39a08edcb45645e9243a40817816ba9e9b2fce7fcb8d1514fa46eee599b4 WatchSource:0}: Error finding container efeb39a08edcb45645e9243a40817816ba9e9b2fce7fcb8d1514fa46eee599b4: Status 404 returned error can't find the container with id efeb39a08edcb45645e9243a40817816ba9e9b2fce7fcb8d1514fa46eee599b4
Jul 26 16:55:37.922949 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:37.922913    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh" event={"ID":"019bdf09-84fb-48ff-9ca8-44bbe17ca127","Type":"ContainerStarted","Data":"2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53"}
Jul 26 16:55:37.922949 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:37.922956    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh" event={"ID":"019bdf09-84fb-48ff-9ca8-44bbe17ca127","Type":"ContainerStarted","Data":"efeb39a08edcb45645e9243a40817816ba9e9b2fce7fcb8d1514fa46eee599b4"}
Jul 26 16:55:37.941109 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:37.941068    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh" podStartSLOduration=1.941053133 podStartE2EDuration="1.941053133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:55:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:55:37.94051272 +0000 UTC m=+2834.123597029" watchObservedRunningTime="2026-07-26 16:55:37.941053133 +0000 UTC m=+2834.124137429"
Jul 26 16:55:40.472899 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:40.472860    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:55:46.889291 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:46.889256    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:46.889754 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:46.889306    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:46.901938 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:46.901909    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:55:46.981029 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:55:46.980999    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:56:10.917904 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:10.917870    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:56:30.923980 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:30.923941    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:56:36.171049 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:36.171015    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"]
Jul 26 16:56:37.174689 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:37.174641    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"]
Jul 26 16:56:49.352208 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:49.352156    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:56:51.190389 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.190347    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b" podUID="a95c7946-fcd8-4b1d-a579-65632ef3b1d6" containerName="main" containerID="cri-o://bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3" gracePeriod=45
Jul 26 16:56:51.441629 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.441570    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:56:51.585338 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.585292    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh2tz\" (UniqueName: \"kubernetes.io/projected/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-kube-api-access-xh2tz\") pod \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") "
Jul 26 16:56:51.585338 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.585337    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-dshm\") pod \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") "
Jul 26 16:56:51.585592 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.585361    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-model-cache\") pod \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") "
Jul 26 16:56:51.585592 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.585385    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-home\") pod \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") "
Jul 26 16:56:51.585592 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.585405    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tmp-dir\") pod \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") "
Jul 26 16:56:51.585592 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.585467    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tls-certs\") pod \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\" (UID: \"a95c7946-fcd8-4b1d-a579-65632ef3b1d6\") "
Jul 26 16:56:51.585802 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.585653    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "a95c7946-fcd8-4b1d-a579-65632ef3b1d6" (UID: "a95c7946-fcd8-4b1d-a579-65632ef3b1d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:56:51.585944 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.585906    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:51.585944 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.585902    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a95c7946-fcd8-4b1d-a579-65632ef3b1d6" (UID: "a95c7946-fcd8-4b1d-a579-65632ef3b1d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:56:51.586063 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.585961    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-home" (OuterVolumeSpecName: "home") pod "a95c7946-fcd8-4b1d-a579-65632ef3b1d6" (UID: "a95c7946-fcd8-4b1d-a579-65632ef3b1d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:56:51.587613 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.587588    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a95c7946-fcd8-4b1d-a579-65632ef3b1d6" (UID: "a95c7946-fcd8-4b1d-a579-65632ef3b1d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:56:51.587739 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.587643    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-dshm" (OuterVolumeSpecName: "dshm") pod "a95c7946-fcd8-4b1d-a579-65632ef3b1d6" (UID: "a95c7946-fcd8-4b1d-a579-65632ef3b1d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:56:51.587854 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.587827    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-kube-api-access-xh2tz" (OuterVolumeSpecName: "kube-api-access-xh2tz") pod "a95c7946-fcd8-4b1d-a579-65632ef3b1d6" (UID: "a95c7946-fcd8-4b1d-a579-65632ef3b1d6"). InnerVolumeSpecName "kube-api-access-xh2tz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:56:51.686607 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.686571    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh2tz\" (UniqueName: \"kubernetes.io/projected/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-kube-api-access-xh2tz\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:51.686607 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.686603    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:51.686607 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.686613    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:51.686847 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.686622    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:51.686847 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:51.686630    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a95c7946-fcd8-4b1d-a579-65632ef3b1d6-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:52.193528 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.193486    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh" podUID="019bdf09-84fb-48ff-9ca8-44bbe17ca127" containerName="main" containerID="cri-o://2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53" gracePeriod=45
Jul 26 16:56:52.267671 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.267636    2570 generic.go:356] "Generic (PLEG): container finished" podID="a95c7946-fcd8-4b1d-a579-65632ef3b1d6" containerID="bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3" exitCode=0
Jul 26 16:56:52.267859 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.267688    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"
Jul 26 16:56:52.267859 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.267721    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b" event={"ID":"a95c7946-fcd8-4b1d-a579-65632ef3b1d6","Type":"ContainerDied","Data":"bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3"}
Jul 26 16:56:52.267859 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.267764    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b" event={"ID":"a95c7946-fcd8-4b1d-a579-65632ef3b1d6","Type":"ContainerDied","Data":"8770049e9bff101c54aa4e37ec7a96a780f2e7d08298c5c6bf812743941e2b60"}
Jul 26 16:56:52.267859 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.267781    2570 scope.go:117] "RemoveContainer" containerID="bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3"
Jul 26 16:56:52.294704 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.294671    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"]
Jul 26 16:56:52.300718 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.300693    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/sub-test-model-1b2072-kserve-7c7c845975-hhh2b"]
Jul 26 16:56:52.304913 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.304891    2570 scope.go:117] "RemoveContainer" containerID="bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3"
Jul 26 16:56:52.305216 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:56:52.305181    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3\": container with ID starting with bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3 not found: ID does not exist" containerID="bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3"
Jul 26 16:56:52.305308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.305221    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3"} err="failed to get container status \"bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3\": rpc error: code = NotFound desc = could not find container \"bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3\": container with ID starting with bbe106d10a6c5deaeae5bd6b0dd1ef93af2aa20bd211e6693a4ac72d397432b3 not found: ID does not exist"
Jul 26 16:56:52.441350 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.441325    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:56:52.594478 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.594446    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-dshm\") pod \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") "
Jul 26 16:56:52.594678 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.594547    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tmp-dir\") pod \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") "
Jul 26 16:56:52.594678 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.594570    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzrgz\" (UniqueName: \"kubernetes.io/projected/019bdf09-84fb-48ff-9ca8-44bbe17ca127-kube-api-access-fzrgz\") pod \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") "
Jul 26 16:56:52.594678 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.594589    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tls-certs\") pod \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") "
Jul 26 16:56:52.594678 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.594606    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-model-cache\") pod \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") "
Jul 26 16:56:52.594678 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.594627    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-home\") pod \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\" (UID: \"019bdf09-84fb-48ff-9ca8-44bbe17ca127\") "
Jul 26 16:56:52.594961 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.594853    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "019bdf09-84fb-48ff-9ca8-44bbe17ca127" (UID: "019bdf09-84fb-48ff-9ca8-44bbe17ca127"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:56:52.594961 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.594929    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-model-cache" (OuterVolumeSpecName: "model-cache") pod "019bdf09-84fb-48ff-9ca8-44bbe17ca127" (UID: "019bdf09-84fb-48ff-9ca8-44bbe17ca127"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:56:52.595133 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.595113    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-home" (OuterVolumeSpecName: "home") pod "019bdf09-84fb-48ff-9ca8-44bbe17ca127" (UID: "019bdf09-84fb-48ff-9ca8-44bbe17ca127"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:56:52.596866 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.596843    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "019bdf09-84fb-48ff-9ca8-44bbe17ca127" (UID: "019bdf09-84fb-48ff-9ca8-44bbe17ca127"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:56:52.597047 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.597032    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-dshm" (OuterVolumeSpecName: "dshm") pod "019bdf09-84fb-48ff-9ca8-44bbe17ca127" (UID: "019bdf09-84fb-48ff-9ca8-44bbe17ca127"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:56:52.597106 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.597065    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/019bdf09-84fb-48ff-9ca8-44bbe17ca127-kube-api-access-fzrgz" (OuterVolumeSpecName: "kube-api-access-fzrgz") pod "019bdf09-84fb-48ff-9ca8-44bbe17ca127" (UID: "019bdf09-84fb-48ff-9ca8-44bbe17ca127"). InnerVolumeSpecName "kube-api-access-fzrgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:56:52.695537 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.695499    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:52.695537 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.695535    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:52.695730 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.695551    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzrgz\" (UniqueName: \"kubernetes.io/projected/019bdf09-84fb-48ff-9ca8-44bbe17ca127-kube-api-access-fzrgz\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:52.695730 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.695564    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/019bdf09-84fb-48ff-9ca8-44bbe17ca127-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:52.695730 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.695575    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:52.695730 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:52.695585    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/019bdf09-84fb-48ff-9ca8-44bbe17ca127-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:56:53.273702 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:53.273670    2570 generic.go:356] "Generic (PLEG): container finished" podID="019bdf09-84fb-48ff-9ca8-44bbe17ca127" containerID="2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53" exitCode=0
Jul 26 16:56:53.273702 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:53.273715    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"
Jul 26 16:56:53.274288 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:53.273768    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh" event={"ID":"019bdf09-84fb-48ff-9ca8-44bbe17ca127","Type":"ContainerDied","Data":"2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53"}
Jul 26 16:56:53.274288 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:53.273819    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh" event={"ID":"019bdf09-84fb-48ff-9ca8-44bbe17ca127","Type":"ContainerDied","Data":"efeb39a08edcb45645e9243a40817816ba9e9b2fce7fcb8d1514fa46eee599b4"}
Jul 26 16:56:53.274288 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:53.273844    2570 scope.go:117] "RemoveContainer" containerID="2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53"
Jul 26 16:56:53.282555 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:53.282538    2570 scope.go:117] "RemoveContainer" containerID="2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53"
Jul 26 16:56:53.282816 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:56:53.282791    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53\": container with ID starting with 2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53 not found: ID does not exist" containerID="2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53"
Jul 26 16:56:53.282861 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:53.282823    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53"} err="failed to get container status \"2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53\": rpc error: code = NotFound desc = could not find container \"2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53\": container with ID starting with 2b3d8b99b18d08ebf3b4dc01e8e56eb6caf5909548de5347785adb721cddcf53 not found: ID does not exist"
Jul 26 16:56:53.296709 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:53.296682    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"]
Jul 26 16:56:53.302753 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:53.302731    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a06722/sub-test-model-a06722-kserve-55bc45655f-r5rmh"]
Jul 26 16:56:54.300086 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:54.300055    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="019bdf09-84fb-48ff-9ca8-44bbe17ca127" path="/var/lib/kubelet/pods/019bdf09-84fb-48ff-9ca8-44bbe17ca127/volumes"
Jul 26 16:56:54.300483 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:56:54.300453    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a95c7946-fcd8-4b1d-a579-65632ef3b1d6" path="/var/lib/kubelet/pods/a95c7946-fcd8-4b1d-a579-65632ef3b1d6/volumes"
Jul 26 16:57:03.672320 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.672279    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"]
Jul 26 16:57:03.673010 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.672988    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a95c7946-fcd8-4b1d-a579-65632ef3b1d6" containerName="main"
Jul 26 16:57:03.673084 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.673015    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a95c7946-fcd8-4b1d-a579-65632ef3b1d6" containerName="main"
Jul 26 16:57:03.673084 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.673073    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="019bdf09-84fb-48ff-9ca8-44bbe17ca127" containerName="main"
Jul 26 16:57:03.673084 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.673083    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="019bdf09-84fb-48ff-9ca8-44bbe17ca127" containerName="main"
Jul 26 16:57:03.673303 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.673264    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a95c7946-fcd8-4b1d-a579-65632ef3b1d6" containerName="main"
Jul 26 16:57:03.673303 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.673289    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="019bdf09-84fb-48ff-9ca8-44bbe17ca127" containerName="main"
Jul 26 16:57:03.678354 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.678334    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.680881 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.680853    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-home\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.680972 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.680915    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-model-cache\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.680972 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.680945    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-dshm\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.680972 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.680962    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-479cr\" (UniqueName: \"kubernetes.io/projected/80b44854-bafe-44f5-9647-6e9d18351057-kube-api-access-479cr\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.681078 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.680983    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-tmp-dir\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.681078 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.681003    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80b44854-bafe-44f5-9647-6e9d18351057-tls-certs\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.684406 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.684384    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"rate-test-model-1b2072-kserve-self-signed-certs\""
Jul 26 16:57:03.684528 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.684443    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"default-dockercfg-5r7cf\""
Jul 26 16:57:03.684911 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.684624    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"kube-root-ca.crt\""
Jul 26 16:57:03.684911 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.684654    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1b2072\"/\"openshift-service-ca.crt\""
Jul 26 16:57:03.721645 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.721612    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"]
Jul 26 16:57:03.781996 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.781956    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-model-cache\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.781996 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.781996    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-dshm\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.782274 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.782012    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-479cr\" (UniqueName: \"kubernetes.io/projected/80b44854-bafe-44f5-9647-6e9d18351057-kube-api-access-479cr\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.782274 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.782032    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-tmp-dir\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.782274 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.782049    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80b44854-bafe-44f5-9647-6e9d18351057-tls-certs\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.782274 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.782116    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-home\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.782469 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.782383    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-model-cache\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.782469 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.782434    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-tmp-dir\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.782549 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.782529    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-home\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.784326 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.784306    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-dshm\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.784590 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.784572    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80b44854-bafe-44f5-9647-6e9d18351057-tls-certs\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.803299 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.803267    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-479cr\" (UniqueName: \"kubernetes.io/projected/80b44854-bafe-44f5-9647-6e9d18351057-kube-api-access-479cr\") pod \"rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") " pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:03.987247 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:03.987205    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:04.132216 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:04.132170    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"]
Jul 26 16:57:04.135807 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:57:04.135780    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80b44854_bafe_44f5_9647_6e9d18351057.slice/crio-323790f62553f16c1a2b1c139ac71a357e492bbb54ad238513d0ee865a217bc7 WatchSource:0}: Error finding container 323790f62553f16c1a2b1c139ac71a357e492bbb54ad238513d0ee865a217bc7: Status 404 returned error can't find the container with id 323790f62553f16c1a2b1c139ac71a357e492bbb54ad238513d0ee865a217bc7
Jul 26 16:57:04.325874 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:04.325794    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t" event={"ID":"80b44854-bafe-44f5-9647-6e9d18351057","Type":"ContainerStarted","Data":"17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574"}
Jul 26 16:57:04.325874 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:04.325843    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t" event={"ID":"80b44854-bafe-44f5-9647-6e9d18351057","Type":"ContainerStarted","Data":"323790f62553f16c1a2b1c139ac71a357e492bbb54ad238513d0ee865a217bc7"}
Jul 26 16:57:04.355378 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:04.355329    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t" podStartSLOduration=1.355311838 podStartE2EDuration="1.355311838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:57:04.354000307 +0000 UTC m=+2920.537084594" watchObservedRunningTime="2026-07-26 16:57:04.355311838 +0000 UTC m=+2920.538396103"
Jul 26 16:57:07.139032 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:07.138999    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:57:13.987627 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:13.987581    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:13.988030 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:13.987674    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:14.002439 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:14.002412    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:14.383724 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:14.383634    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:57:35.524165 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.524128    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"]
Jul 26 16:57:35.527778 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.527757    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.531322 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.531298    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a06722\"/\"rate-test-model-a06722-kserve-self-signed-certs\""
Jul 26 16:57:35.532674 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.532654    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a06722\"/\"kube-root-ca.crt\""
Jul 26 16:57:35.532791 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.532654    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a06722\"/\"default-dockercfg-4bmgt\""
Jul 26 16:57:35.532791 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.532707    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a06722\"/\"openshift-service-ca.crt\""
Jul 26 16:57:35.561745 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.561714    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"]
Jul 26 16:57:35.671084 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.671044    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frvzz\" (UniqueName: \"kubernetes.io/projected/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-kube-api-access-frvzz\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.671305 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.671089    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-dshm\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.671305 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.671115    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-model-cache\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.671305 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.671245    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-home\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.671437 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.671309    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tmp-dir\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.671437 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.671334    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.772121 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.772079    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-home\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.772329 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.772148    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tmp-dir\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.772329 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.772175    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.772329 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.772248    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frvzz\" (UniqueName: \"kubernetes.io/projected/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-kube-api-access-frvzz\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.772329 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.772278    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-dshm\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.772329 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.772323    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-model-cache\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.772611 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.772547    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-home\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.772611 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.772604    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tmp-dir\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.772725 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.772654    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-model-cache\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.774624 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.774565    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-dshm\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.774789 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.774773    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.783308 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.783288    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frvzz\" (UniqueName: \"kubernetes.io/projected/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-kube-api-access-frvzz\") pod \"rate-test-model-a06722-kserve-768849b675-n5kcr\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") " pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.837149 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.837115    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:35.989049 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:35.988893    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"]
Jul 26 16:57:35.991655 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:57:35.991629    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd65ab1c5_35f9_4a99_ad55_d9ff99c2e20b.slice/crio-4f89fe2111f798d8ace6491d48ff2e2c626baf99a0a6b794bdfeda57df6f189b WatchSource:0}: Error finding container 4f89fe2111f798d8ace6491d48ff2e2c626baf99a0a6b794bdfeda57df6f189b: Status 404 returned error can't find the container with id 4f89fe2111f798d8ace6491d48ff2e2c626baf99a0a6b794bdfeda57df6f189b
Jul 26 16:57:36.473430 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:36.473389    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr" event={"ID":"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b","Type":"ContainerStarted","Data":"a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652"}
Jul 26 16:57:36.473430 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:36.473430    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr" event={"ID":"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b","Type":"ContainerStarted","Data":"4f89fe2111f798d8ace6491d48ff2e2c626baf99a0a6b794bdfeda57df6f189b"}
Jul 26 16:57:36.502617 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:36.502560    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr" podStartSLOduration=1.5025437670000001 podStartE2EDuration="1.502543767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:57:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:57:36.500869257 +0000 UTC m=+2952.683953543" watchObservedRunningTime="2026-07-26 16:57:36.502543767 +0000 UTC m=+2952.685628053"
Jul 26 16:57:39.657921 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:39.657883    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:57:45.837311 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:45.837279    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:45.837706 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:45.837342    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:45.849433 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:45.849409    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:57:46.530489 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:57:46.530405    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:58:21.749570 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:58:21.749534    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:58:24.441828 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:58:24.441791    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:58:24.445348 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:58:24.445326    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 16:58:36.128270 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:58:36.128239    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:58:51.420179 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:58:51.420135    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:58:55.488086 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:58:55.488052    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"]
Jul 26 16:58:56.304092 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:58:56.304062    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"]
Jul 26 16:58:56.387535 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:58:56.387294    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-self-signed-certs: secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:58:56.387535 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:58:56.387386    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs podName:d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b nodeName:}" failed. No retries permitted until 2026-07-26 16:58:56.887362894 +0000 UTC m=+3033.070447182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs") pod "rate-test-model-a06722-kserve-768849b675-n5kcr" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b") : secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:58:56.892526 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:58:56.892495    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-self-signed-certs: secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:58:56.892970 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:58:56.892561    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs podName:d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b nodeName:}" failed. No retries permitted until 2026-07-26 16:58:57.892545415 +0000 UTC m=+3034.075629680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs") pod "rate-test-model-a06722-kserve-768849b675-n5kcr" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b") : secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:58:57.902241 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:58:57.902204    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-self-signed-certs: secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:58:57.902620 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:58:57.902275    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs podName:d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b nodeName:}" failed. No retries permitted until 2026-07-26 16:58:59.902259848 +0000 UTC m=+3036.085344112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs") pod "rate-test-model-a06722-kserve-768849b675-n5kcr" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b") : secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:58:59.923730 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:58:59.923690    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-self-signed-certs: secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:58:59.924128 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:58:59.923766    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs podName:d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b nodeName:}" failed. No retries permitted until 2026-07-26 16:59:03.923752313 +0000 UTC m=+3040.106836579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs") pod "rate-test-model-a06722-kserve-768849b675-n5kcr" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b") : secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:59:03.961240 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:59:03.961182    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-self-signed-certs: secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:59:03.961696 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:59:03.961275    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs podName:d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b nodeName:}" failed. No retries permitted until 2026-07-26 16:59:11.961261411 +0000 UTC m=+3048.144345676 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs") pod "rate-test-model-a06722-kserve-768849b675-n5kcr" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b") : secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:59:07.819949 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:07.819911    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:59:10.506493 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.506435    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t" podUID="80b44854-bafe-44f5-9647-6e9d18351057" containerName="main" containerID="cri-o://17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574" gracePeriod=45
Jul 26 16:59:10.757271 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.757179    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:59:10.896384 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.896349    2570 generic.go:356] "Generic (PLEG): container finished" podID="80b44854-bafe-44f5-9647-6e9d18351057" containerID="17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574" exitCode=0
Jul 26 16:59:10.896544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.896398    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"
Jul 26 16:59:10.896544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.896435    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t" event={"ID":"80b44854-bafe-44f5-9647-6e9d18351057","Type":"ContainerDied","Data":"17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574"}
Jul 26 16:59:10.896544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.896479    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t" event={"ID":"80b44854-bafe-44f5-9647-6e9d18351057","Type":"ContainerDied","Data":"323790f62553f16c1a2b1c139ac71a357e492bbb54ad238513d0ee865a217bc7"}
Jul 26 16:59:10.896544 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.896499    2570 scope.go:117] "RemoveContainer" containerID="17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574"
Jul 26 16:59:10.904559 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.904550    2570 scope.go:117] "RemoveContainer" containerID="17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574"
Jul 26 16:59:10.904817 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:59:10.904796    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574\": container with ID starting with 17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574 not found: ID does not exist" containerID="17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574"
Jul 26 16:59:10.904886 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.904829    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574"} err="failed to get container status \"17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574\": rpc error: code = NotFound desc = could not find container \"17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574\": container with ID starting with 17a7c843fb64df1520712fa94d1e0d5ebed99369e6d0e7a448f217bcb01b7574 not found: ID does not exist"
Jul 26 16:59:10.930138 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.930120    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-home\") pod \"80b44854-bafe-44f5-9647-6e9d18351057\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") "
Jul 26 16:59:10.930216 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.930149    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-479cr\" (UniqueName: \"kubernetes.io/projected/80b44854-bafe-44f5-9647-6e9d18351057-kube-api-access-479cr\") pod \"80b44854-bafe-44f5-9647-6e9d18351057\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") "
Jul 26 16:59:10.930216 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.930169    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-dshm\") pod \"80b44854-bafe-44f5-9647-6e9d18351057\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") "
Jul 26 16:59:10.930216 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.930203    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80b44854-bafe-44f5-9647-6e9d18351057-tls-certs\") pod \"80b44854-bafe-44f5-9647-6e9d18351057\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") "
Jul 26 16:59:10.930374 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.930238    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-model-cache\") pod \"80b44854-bafe-44f5-9647-6e9d18351057\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") "
Jul 26 16:59:10.930374 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.930258    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-tmp-dir\") pod \"80b44854-bafe-44f5-9647-6e9d18351057\" (UID: \"80b44854-bafe-44f5-9647-6e9d18351057\") "
Jul 26 16:59:10.930519 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.930490    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-model-cache" (OuterVolumeSpecName: "model-cache") pod "80b44854-bafe-44f5-9647-6e9d18351057" (UID: "80b44854-bafe-44f5-9647-6e9d18351057"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:59:10.930657 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.930566    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-home" (OuterVolumeSpecName: "home") pod "80b44854-bafe-44f5-9647-6e9d18351057" (UID: "80b44854-bafe-44f5-9647-6e9d18351057"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:59:10.930732 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.930646    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80b44854-bafe-44f5-9647-6e9d18351057" (UID: "80b44854-bafe-44f5-9647-6e9d18351057"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:59:10.932256 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.932231    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80b44854-bafe-44f5-9647-6e9d18351057-kube-api-access-479cr" (OuterVolumeSpecName: "kube-api-access-479cr") pod "80b44854-bafe-44f5-9647-6e9d18351057" (UID: "80b44854-bafe-44f5-9647-6e9d18351057"). InnerVolumeSpecName "kube-api-access-479cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:59:10.932356 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.932336    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-dshm" (OuterVolumeSpecName: "dshm") pod "80b44854-bafe-44f5-9647-6e9d18351057" (UID: "80b44854-bafe-44f5-9647-6e9d18351057"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:59:10.932416 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:10.932397    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80b44854-bafe-44f5-9647-6e9d18351057-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80b44854-bafe-44f5-9647-6e9d18351057" (UID: "80b44854-bafe-44f5-9647-6e9d18351057"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:59:11.031788 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:11.031707    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:11.031788 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:11.031736    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-479cr\" (UniqueName: \"kubernetes.io/projected/80b44854-bafe-44f5-9647-6e9d18351057-kube-api-access-479cr\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:11.031788 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:11.031747    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:11.031788 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:11.031755    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80b44854-bafe-44f5-9647-6e9d18351057-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:11.031788 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:11.031764    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:11.031788 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:11.031772    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80b44854-bafe-44f5-9647-6e9d18351057-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:11.220038 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:11.220007    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"]
Jul 26 16:59:11.227228 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:11.227174    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1b2072/rate-test-model-1b2072-kserve-86cfb8fbc9-wql7t"]
Jul 26 16:59:11.848180 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:11.848140    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr" podUID="d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" containerName="main" containerID="cri-o://a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652" gracePeriod=45
Jul 26 16:59:12.039982 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:59:12.039955    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-self-signed-certs: secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:59:12.040113 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:59:12.040029    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs podName:d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b nodeName:}" failed. No retries permitted until 2026-07-26 16:59:28.040011691 +0000 UTC m=+3064.223095956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs") pod "rate-test-model-a06722-kserve-768849b675-n5kcr" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b") : secret "rate-test-model-a06722-kserve-self-signed-certs" not found
Jul 26 16:59:12.093490 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.093464    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:59:12.140597 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.140540    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs\") pod \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") "
Jul 26 16:59:12.140597 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.140580    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-dshm\") pod \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") "
Jul 26 16:59:12.140764 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.140603    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frvzz\" (UniqueName: \"kubernetes.io/projected/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-kube-api-access-frvzz\") pod \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") "
Jul 26 16:59:12.140764 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.140622    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-home\") pod \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") "
Jul 26 16:59:12.140764 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.140649    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-model-cache\") pod \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") "
Jul 26 16:59:12.140764 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.140674    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tmp-dir\") pod \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\" (UID: \"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b\") "
Jul 26 16:59:12.141280 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.140934    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-home" (OuterVolumeSpecName: "home") pod "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:59:12.141280 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.141002    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-model-cache" (OuterVolumeSpecName: "model-cache") pod "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:59:12.141280 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.141039    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:59:12.143214 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.143170    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-dshm" (OuterVolumeSpecName: "dshm") pod "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:59:12.143327 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.143232    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:59:12.143327 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.143256    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-kube-api-access-frvzz" (OuterVolumeSpecName: "kube-api-access-frvzz") pod "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" (UID: "d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b"). InnerVolumeSpecName "kube-api-access-frvzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:59:12.241934 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.241890    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:12.241934 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.241928    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:12.241934 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.241939    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frvzz\" (UniqueName: \"kubernetes.io/projected/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-kube-api-access-frvzz\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:12.241934 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.241948    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:12.242180 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.241956    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:12.242180 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.241964    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:12.301039 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.301003    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80b44854-bafe-44f5-9647-6e9d18351057" path="/var/lib/kubelet/pods/80b44854-bafe-44f5-9647-6e9d18351057/volumes"
Jul 26 16:59:12.783309 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.783275    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7391-139635694012000"]
Jul 26 16:59:12.783630 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.783617    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80b44854-bafe-44f5-9647-6e9d18351057" containerName="main"
Jul 26 16:59:12.783682 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.783632    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="80b44854-bafe-44f5-9647-6e9d18351057" containerName="main"
Jul 26 16:59:12.783682 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.783648    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" containerName="main"
Jul 26 16:59:12.783682 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.783654    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" containerName="main"
Jul 26 16:59:12.783774 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.783701    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" containerName="main"
Jul 26 16:59:12.783774 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.783712    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="80b44854-bafe-44f5-9647-6e9d18351057" containerName="main"
Jul 26 16:59:12.786663 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.786647    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-139635694012000"
Jul 26 16:59:12.790350 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.790334    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4rt52\""
Jul 26 16:59:12.800640 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.800617    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-139635694012000"]
Jul 26 16:59:12.846178 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.846146    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v9ws\" (UniqueName: \"kubernetes.io/projected/d7f590de-20d7-42b0-ae07-bbe3daaeb098-kube-api-access-9v9ws\") pod \"test-curl-7391-139635694012000\" (UID: \"d7f590de-20d7-42b0-ae07-bbe3daaeb098\") " pod="opendatahub/test-curl-7391-139635694012000"
Jul 26 16:59:12.906673 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.906640    2570 generic.go:356] "Generic (PLEG): container finished" podID="d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" containerID="a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652" exitCode=0
Jul 26 16:59:12.907157 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.906685    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr" event={"ID":"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b","Type":"ContainerDied","Data":"a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652"}
Jul 26 16:59:12.907157 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.906693    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"
Jul 26 16:59:12.907157 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.906713    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr" event={"ID":"d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b","Type":"ContainerDied","Data":"4f89fe2111f798d8ace6491d48ff2e2c626baf99a0a6b794bdfeda57df6f189b"}
Jul 26 16:59:12.907157 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.906731    2570 scope.go:117] "RemoveContainer" containerID="a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652"
Jul 26 16:59:12.914869 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.914842    2570 scope.go:117] "RemoveContainer" containerID="a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652"
Jul 26 16:59:12.915143 ip-10-0-135-132 kubenswrapper[2570]: E0726 16:59:12.915124    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652\": container with ID starting with a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652 not found: ID does not exist" containerID="a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652"
Jul 26 16:59:12.915209 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.915152    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652"} err="failed to get container status \"a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652\": rpc error: code = NotFound desc = could not find container \"a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652\": container with ID starting with a7ba65539085cb89dea014794828661acee96e5c9b9b3c447894f99136a3e652 not found: ID does not exist"
Jul 26 16:59:12.928627 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.928604    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"]
Jul 26 16:59:12.932887 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.932863    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a06722/rate-test-model-a06722-kserve-768849b675-n5kcr"]
Jul 26 16:59:12.946814 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.946784    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9ws\" (UniqueName: \"kubernetes.io/projected/d7f590de-20d7-42b0-ae07-bbe3daaeb098-kube-api-access-9v9ws\") pod \"test-curl-7391-139635694012000\" (UID: \"d7f590de-20d7-42b0-ae07-bbe3daaeb098\") " pod="opendatahub/test-curl-7391-139635694012000"
Jul 26 16:59:12.955040 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:12.955020    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v9ws\" (UniqueName: \"kubernetes.io/projected/d7f590de-20d7-42b0-ae07-bbe3daaeb098-kube-api-access-9v9ws\") pod \"test-curl-7391-139635694012000\" (UID: \"d7f590de-20d7-42b0-ae07-bbe3daaeb098\") " pod="opendatahub/test-curl-7391-139635694012000"
Jul 26 16:59:13.094852 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:13.094766    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-139635694012000"
Jul 26 16:59:13.226565 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:13.226542    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-139635694012000"]
Jul 26 16:59:13.228773 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:59:13.228747    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7f590de_20d7_42b0_ae07_bbe3daaeb098.slice/crio-eb566c4dab271bddd4a3ead986703277ac6e8c17de43cdac675d8c8aee2e6820 WatchSource:0}: Error finding container eb566c4dab271bddd4a3ead986703277ac6e8c17de43cdac675d8c8aee2e6820: Status 404 returned error can't find the container with id eb566c4dab271bddd4a3ead986703277ac6e8c17de43cdac675d8c8aee2e6820
Jul 26 16:59:13.230106 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:13.230084    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:59:13.912013 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:13.911984    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-139635694012000" event={"ID":"d7f590de-20d7-42b0-ae07-bbe3daaeb098","Type":"ContainerStarted","Data":"eb566c4dab271bddd4a3ead986703277ac6e8c17de43cdac675d8c8aee2e6820"}
Jul 26 16:59:14.302075 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:14.302036    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b" path="/var/lib/kubelet/pods/d65ab1c5-35f9-4a99-ad55-d9ff99c2e20b/volumes"
Jul 26 16:59:14.918567 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:14.918529    2570 generic.go:356] "Generic (PLEG): container finished" podID="d7f590de-20d7-42b0-ae07-bbe3daaeb098" containerID="6648bff5f8ab093dc5b05e714d2f2373d635d45911e0737713fecf3f51bf62fa" exitCode=0
Jul 26 16:59:14.918961 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:14.918616    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-139635694012000" event={"ID":"d7f590de-20d7-42b0-ae07-bbe3daaeb098","Type":"ContainerDied","Data":"6648bff5f8ab093dc5b05e714d2f2373d635d45911e0737713fecf3f51bf62fa"}
Jul 26 16:59:16.051515 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:16.051490    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-139635694012000"
Jul 26 16:59:16.073841 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:16.073817    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v9ws\" (UniqueName: \"kubernetes.io/projected/d7f590de-20d7-42b0-ae07-bbe3daaeb098-kube-api-access-9v9ws\") pod \"d7f590de-20d7-42b0-ae07-bbe3daaeb098\" (UID: \"d7f590de-20d7-42b0-ae07-bbe3daaeb098\") "
Jul 26 16:59:16.075853 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:16.075826    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7f590de-20d7-42b0-ae07-bbe3daaeb098-kube-api-access-9v9ws" (OuterVolumeSpecName: "kube-api-access-9v9ws") pod "d7f590de-20d7-42b0-ae07-bbe3daaeb098" (UID: "d7f590de-20d7-42b0-ae07-bbe3daaeb098"). InnerVolumeSpecName "kube-api-access-9v9ws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:59:16.115179 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:16.115156    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7391-139635694012000_d7f590de-20d7-42b0-ae07-bbe3daaeb098/test-curl-7391-139635694012000/0.log"
Jul 26 16:59:16.143408 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:16.143382    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7391-139635694012000"]
Jul 26 16:59:16.149842 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:16.149820    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7391-139635694012000"]
Jul 26 16:59:16.175287 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:16.175231    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v9ws\" (UniqueName: \"kubernetes.io/projected/d7f590de-20d7-42b0-ae07-bbe3daaeb098-kube-api-access-9v9ws\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 16:59:16.300404 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:16.300369    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7f590de-20d7-42b0-ae07-bbe3daaeb098" path="/var/lib/kubelet/pods/d7f590de-20d7-42b0-ae07-bbe3daaeb098/volumes"
Jul 26 16:59:16.929770 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:16.929579    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-139635694012000"
Jul 26 16:59:16.929770 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:16.929611    2570 scope.go:117] "RemoveContainer" containerID="6648bff5f8ab093dc5b05e714d2f2373d635d45911e0737713fecf3f51bf62fa"
Jul 26 16:59:29.138598 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:29.138562    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:59:46.333800 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.333765    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"]
Jul 26 16:59:46.334274 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.334082    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f590de-20d7-42b0-ae07-bbe3daaeb098" containerName="test-curl-7391-139635694012000"
Jul 26 16:59:46.334274 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.334093    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f590de-20d7-42b0-ae07-bbe3daaeb098" containerName="test-curl-7391-139635694012000"
Jul 26 16:59:46.334274 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.334152    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f590de-20d7-42b0-ae07-bbe3daaeb098" containerName="test-curl-7391-139635694012000"
Jul 26 16:59:46.337361 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.337342    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.340987 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.340957    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-930b99-serving-cert\""
Jul 26 16:59:46.341130 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.341071    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c6ndf\""
Jul 26 16:59:46.341130 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.341103    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 16:59:46.342387 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.342369    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 16:59:46.352311 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.352289    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"]
Jul 26 16:59:46.444518 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.444479    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnjwv\" (UniqueName: \"kubernetes.io/projected/f742be70-47f3-4523-8f52-aacc075b10fa-kube-api-access-fnjwv\") pod \"maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.444687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.444586    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f742be70-47f3-4523-8f52-aacc075b10fa-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.444687 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.444646    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f742be70-47f3-4523-8f52-aacc075b10fa-maas-api-tls\") pod \"maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.545467 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.545435    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnjwv\" (UniqueName: \"kubernetes.io/projected/f742be70-47f3-4523-8f52-aacc075b10fa-kube-api-access-fnjwv\") pod \"maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.545614 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.545482    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f742be70-47f3-4523-8f52-aacc075b10fa-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.545614 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.545524    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f742be70-47f3-4523-8f52-aacc075b10fa-maas-api-tls\") pod \"maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.546297 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.546272    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f742be70-47f3-4523-8f52-aacc075b10fa-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.547914 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.547897    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f742be70-47f3-4523-8f52-aacc075b10fa-maas-api-tls\") pod \"maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.553194 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.553166    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnjwv\" (UniqueName: \"kubernetes.io/projected/f742be70-47f3-4523-8f52-aacc075b10fa-kube-api-access-fnjwv\") pod \"maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.649667 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.649578    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:46.783954 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.783873    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"]
Jul 26 16:59:46.786922 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:59:46.786892    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf742be70_47f3_4523_8f52_aacc075b10fa.slice/crio-35fcffa8477d384cb161771cd109deacec3c5df225a5dde2d3ccb823bacd4cf0 WatchSource:0}: Error finding container 35fcffa8477d384cb161771cd109deacec3c5df225a5dde2d3ccb823bacd4cf0: Status 404 returned error can't find the container with id 35fcffa8477d384cb161771cd109deacec3c5df225a5dde2d3ccb823bacd4cf0
Jul 26 16:59:46.995013 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.994978    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"]
Jul 26 16:59:46.999289 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:46.999272    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 16:59:47.002532 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.002504    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-930b99\""
Jul 26 16:59:47.002664 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.002511    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-930b99-dockercfg-sbmvp\""
Jul 26 16:59:47.009637 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.009612    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"]
Jul 26 16:59:47.067745 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.067710    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp" event={"ID":"f742be70-47f3-4523-8f52-aacc075b10fa","Type":"ContainerStarted","Data":"35fcffa8477d384cb161771cd109deacec3c5df225a5dde2d3ccb823bacd4cf0"}
Jul 26 16:59:47.151281 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.151249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsnbv\" (UniqueName: \"kubernetes.io/projected/2747ef12-136d-4b51-8c1f-1d876b0d7be4-kube-api-access-rsnbv\") pod \"payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr\" (UID: \"2747ef12-136d-4b51-8c1f-1d876b0d7be4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 16:59:47.151449 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.151294    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2747ef12-136d-4b51-8c1f-1d876b0d7be4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr\" (UID: \"2747ef12-136d-4b51-8c1f-1d876b0d7be4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 16:59:47.252371 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.252286    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsnbv\" (UniqueName: \"kubernetes.io/projected/2747ef12-136d-4b51-8c1f-1d876b0d7be4-kube-api-access-rsnbv\") pod \"payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr\" (UID: \"2747ef12-136d-4b51-8c1f-1d876b0d7be4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 16:59:47.252371 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.252324    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2747ef12-136d-4b51-8c1f-1d876b0d7be4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr\" (UID: \"2747ef12-136d-4b51-8c1f-1d876b0d7be4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 16:59:47.252979 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.252962    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2747ef12-136d-4b51-8c1f-1d876b0d7be4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr\" (UID: \"2747ef12-136d-4b51-8c1f-1d876b0d7be4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 16:59:47.261365 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.261342    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsnbv\" (UniqueName: \"kubernetes.io/projected/2747ef12-136d-4b51-8c1f-1d876b0d7be4-kube-api-access-rsnbv\") pod \"payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr\" (UID: \"2747ef12-136d-4b51-8c1f-1d876b0d7be4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 16:59:47.311512 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.311472    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 16:59:47.480387 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:47.480354    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"]
Jul 26 16:59:47.484552 ip-10-0-135-132 kubenswrapper[2570]: W0726 16:59:47.484523    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2747ef12_136d_4b51_8c1f_1d876b0d7be4.slice/crio-95c24883cde915e85f5cc6e316126a95325d97f64abe4a6b86b25f5fcb33e8ce WatchSource:0}: Error finding container 95c24883cde915e85f5cc6e316126a95325d97f64abe4a6b86b25f5fcb33e8ce: Status 404 returned error can't find the container with id 95c24883cde915e85f5cc6e316126a95325d97f64abe4a6b86b25f5fcb33e8ce
Jul 26 16:59:48.075811 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:48.075766    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr" event={"ID":"2747ef12-136d-4b51-8c1f-1d876b0d7be4","Type":"ContainerStarted","Data":"95c24883cde915e85f5cc6e316126a95325d97f64abe4a6b86b25f5fcb33e8ce"}
Jul 26 16:59:51.096436 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:51.096393    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp" event={"ID":"f742be70-47f3-4523-8f52-aacc075b10fa","Type":"ContainerStarted","Data":"8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142"}
Jul 26 16:59:51.096893 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:51.096466    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 16:59:51.097929 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:51.097905    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr" event={"ID":"2747ef12-136d-4b51-8c1f-1d876b0d7be4","Type":"ContainerStarted","Data":"68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2"}
Jul 26 16:59:51.098063 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:51.098046    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 16:59:51.114698 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:51.114650    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp" podStartSLOduration=1.753281636 podStartE2EDuration="5.114633358s" totalImagesPullingTime="3.361351722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:59:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:59:51.114200768 +0000 UTC m=+3087.297285071" watchObservedRunningTime="2026-07-26 16:59:51.114633358 +0000 UTC m=+3087.297717646"
Jul 26 16:59:51.135423 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:51.135382    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr" podStartSLOduration=2.422868364 podStartE2EDuration="5.135367933s" totalImagesPullingTime="2.712499569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:59:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:59:51.133264646 +0000 UTC m=+3087.316348953" watchObservedRunningTime="2026-07-26 16:59:51.135367933 +0000 UTC m=+3087.318452220"
Jul 26 16:59:54.534629 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:54.534588    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 16:59:57.108141 ip-10-0-135-132 kubenswrapper[2570]: I0726 16:59:57.108108    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 17:00:00.164523 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.164487    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d"]
Jul 26 17:00:00.168098 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.168076    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d"
Jul 26 17:00:00.169318 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.169293    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv"]
Jul 26 17:00:00.170762 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.170744    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2xlcn\""
Jul 26 17:00:00.172987 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.172970    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29"]
Jul 26 17:00:00.173139 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.173084    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv"
Jul 26 17:00:00.176511 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.176033    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf"]
Jul 26 17:00:00.176890 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.176843    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29"
Jul 26 17:00:00.180499 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.180478    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm"]
Jul 26 17:00:00.180708 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.180685    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf"
Jul 26 17:00:00.181871 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.181730    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d"
Jul 26 17:00:00.184831 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.184810    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d"]
Jul 26 17:00:00.184985 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.184970    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29"]
Jul 26 17:00:00.188433 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.185094    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv"]
Jul 26 17:00:00.188433 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.185112    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm"
Jul 26 17:00:00.188433 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.186881    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf"]
Jul 26 17:00:00.193812 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.193795    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv"
Jul 26 17:00:00.216241 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.216201    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm"]
Jul 26 17:00:00.232331 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.230491    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29"
Jul 26 17:00:00.240973 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.240949    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf"
Jul 26 17:00:00.275868 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.275841    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm"
Jul 26 17:00:00.439966 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.439901    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29"]
Jul 26 17:00:00.442122 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:00:00.442092    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod677c510a_71d1_4d79_8f5d_0ec0b5c82184.slice/crio-9c09d55a41e8080a7242f95609ea0d8f480e05f19e69c69d874c564ae88b06c6 WatchSource:0}: Error finding container 9c09d55a41e8080a7242f95609ea0d8f480e05f19e69c69d874c564ae88b06c6: Status 404 returned error can't find the container with id 9c09d55a41e8080a7242f95609ea0d8f480e05f19e69c69d874c564ae88b06c6
Jul 26 17:00:00.463023 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.462936    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf"]
Jul 26 17:00:00.467040 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:00:00.466310    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf285656e_c485_4685_8c9d_62c097c187fe.slice/crio-73edefca23037211b127fc5419f910ff44e52f7dac93825a92d962de1b90c0f4 WatchSource:0}: Error finding container 73edefca23037211b127fc5419f910ff44e52f7dac93825a92d962de1b90c0f4: Status 404 returned error can't find the container with id 73edefca23037211b127fc5419f910ff44e52f7dac93825a92d962de1b90c0f4
Jul 26 17:00:00.490977 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.490952    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm"]
Jul 26 17:00:00.507428 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:00:00.507399    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e81ed2_1773_4760_989a_ff81685d0426.slice/crio-dddf8e783b31e9106b9964070a25718c1e0a0a954ac3565f1cd5362e509d6028 WatchSource:0}: Error finding container dddf8e783b31e9106b9964070a25718c1e0a0a954ac3565f1cd5362e509d6028: Status 404 returned error can't find the container with id dddf8e783b31e9106b9964070a25718c1e0a0a954ac3565f1cd5362e509d6028
Jul 26 17:00:00.572104 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.572078    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d"]
Jul 26 17:00:00.574538 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:00:00.574507    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b7f7f4_6a2f_4c54_8a7b_fd8ed76610f7.slice/crio-60620c794bdfa2698d3fd423c15d631cd4dadfa1001c8dd962a7bb8daa685f26 WatchSource:0}: Error finding container 60620c794bdfa2698d3fd423c15d631cd4dadfa1001c8dd962a7bb8daa685f26: Status 404 returned error can't find the container with id 60620c794bdfa2698d3fd423c15d631cd4dadfa1001c8dd962a7bb8daa685f26
Jul 26 17:00:00.613348 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:00.613320    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv"]
Jul 26 17:00:00.614781 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:00:00.614750    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8e02d9_65ec_4807_850d_786ddeb247e5.slice/crio-67678d58bfb81d720e8779174892140af5db4c8310c1f819f465ef623a3979df WatchSource:0}: Error finding container 67678d58bfb81d720e8779174892140af5db4c8310c1f819f465ef623a3979df: Status 404 returned error can't find the container with id 67678d58bfb81d720e8779174892140af5db4c8310c1f819f465ef623a3979df
Jul 26 17:00:01.143112 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.143077    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29" event={"ID":"677c510a-71d1-4d79-8f5d-0ec0b5c82184","Type":"ContainerStarted","Data":"985f401313a7c76a65dcc2800fac779fef53631748b6d0eb32d09c1378e40ee5"}
Jul 26 17:00:01.143338 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.143129    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29" event={"ID":"677c510a-71d1-4d79-8f5d-0ec0b5c82184","Type":"ContainerStarted","Data":"9c09d55a41e8080a7242f95609ea0d8f480e05f19e69c69d874c564ae88b06c6"}
Jul 26 17:00:01.144605 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.144578    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d" event={"ID":"e6b7f7f4-6a2f-4c54-8a7b-fd8ed76610f7","Type":"ContainerStarted","Data":"6d3706d33cf3fd93154e42d96c02c213d6f60c1407609eb6f765aa1dff2d2066"}
Jul 26 17:00:01.144713 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.144629    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d" event={"ID":"e6b7f7f4-6a2f-4c54-8a7b-fd8ed76610f7","Type":"ContainerStarted","Data":"60620c794bdfa2698d3fd423c15d631cd4dadfa1001c8dd962a7bb8daa685f26"}
Jul 26 17:00:01.145982 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.145955    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm" event={"ID":"a4e81ed2-1773-4760-989a-ff81685d0426","Type":"ContainerStarted","Data":"a4255fa8c54f62c0392985a385c0388fad95cc906bf0d9796e4aa2fa83ac55b3"}
Jul 26 17:00:01.146087 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.145994    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm" event={"ID":"a4e81ed2-1773-4760-989a-ff81685d0426","Type":"ContainerStarted","Data":"dddf8e783b31e9106b9964070a25718c1e0a0a954ac3565f1cd5362e509d6028"}
Jul 26 17:00:01.147481 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.147459    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf" event={"ID":"f285656e-c485-4685-8c9d-62c097c187fe","Type":"ContainerStarted","Data":"dfecdd51195af518544069263d432fe08bde347e7bb48a496562adf7f7be4f35"}
Jul 26 17:00:01.147578 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.147496    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf" event={"ID":"f285656e-c485-4685-8c9d-62c097c187fe","Type":"ContainerStarted","Data":"73edefca23037211b127fc5419f910ff44e52f7dac93825a92d962de1b90c0f4"}
Jul 26 17:00:01.148869 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.148850    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv" event={"ID":"0b8e02d9-65ec-4807-850d-786ddeb247e5","Type":"ContainerStarted","Data":"d89c6d4f61e638cd9e8e38eda3efbd7e3758a96420765a185d4e90c1b65ae048"}
Jul 26 17:00:01.148935 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.148887    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv" event={"ID":"0b8e02d9-65ec-4807-850d-786ddeb247e5","Type":"ContainerStarted","Data":"67678d58bfb81d720e8779174892140af5db4c8310c1f819f465ef623a3979df"}
Jul 26 17:00:01.161505 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.161455    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29" podStartSLOduration=1.161441429 podStartE2EDuration="1.161441429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:00:01.159356281 +0000 UTC m=+3097.342440567" watchObservedRunningTime="2026-07-26 17:00:01.161441429 +0000 UTC m=+3097.344525715"
Jul 26 17:00:01.177056 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.177009    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d" podStartSLOduration=1.176996311 podStartE2EDuration="1.176996311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:00:01.176994239 +0000 UTC m=+3097.360078526" watchObservedRunningTime="2026-07-26 17:00:01.176996311 +0000 UTC m=+3097.360080598"
Jul 26 17:00:01.195825 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.195767    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm" podStartSLOduration=1.195747462 podStartE2EDuration="1.195747462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:00:01.194529692 +0000 UTC m=+3097.377613999" watchObservedRunningTime="2026-07-26 17:00:01.195747462 +0000 UTC m=+3097.378831791"
Jul 26 17:00:01.213849 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.213796    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv" podStartSLOduration=1.213777792 podStartE2EDuration="1.213777792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:00:01.211716235 +0000 UTC m=+3097.394800548" watchObservedRunningTime="2026-07-26 17:00:01.213777792 +0000 UTC m=+3097.396862079"
Jul 26 17:00:01.229133 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:01.229084    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf" podStartSLOduration=1.229069341 podStartE2EDuration="1.229069341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:00:01.22764122 +0000 UTC m=+3097.410725509" watchObservedRunningTime="2026-07-26 17:00:01.229069341 +0000 UTC m=+3097.412153628"
Jul 26 17:00:02.104779 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.104699    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 17:00:02.156398 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.156367    2570 generic.go:356] "Generic (PLEG): container finished" podID="0b8e02d9-65ec-4807-850d-786ddeb247e5" containerID="d89c6d4f61e638cd9e8e38eda3efbd7e3758a96420765a185d4e90c1b65ae048" exitCode=0
Jul 26 17:00:02.156628 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.156456    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv" event={"ID":"0b8e02d9-65ec-4807-850d-786ddeb247e5","Type":"ContainerDied","Data":"d89c6d4f61e638cd9e8e38eda3efbd7e3758a96420765a185d4e90c1b65ae048"}
Jul 26 17:00:02.157892 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.157871    2570 generic.go:356] "Generic (PLEG): container finished" podID="677c510a-71d1-4d79-8f5d-0ec0b5c82184" containerID="985f401313a7c76a65dcc2800fac779fef53631748b6d0eb32d09c1378e40ee5" exitCode=0
Jul 26 17:00:02.157979 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.157949    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29" event={"ID":"677c510a-71d1-4d79-8f5d-0ec0b5c82184","Type":"ContainerDied","Data":"985f401313a7c76a65dcc2800fac779fef53631748b6d0eb32d09c1378e40ee5"}
Jul 26 17:00:02.159284 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.159265    2570 generic.go:356] "Generic (PLEG): container finished" podID="e6b7f7f4-6a2f-4c54-8a7b-fd8ed76610f7" containerID="6d3706d33cf3fd93154e42d96c02c213d6f60c1407609eb6f765aa1dff2d2066" exitCode=0
Jul 26 17:00:02.159375 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.159336    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d" event={"ID":"e6b7f7f4-6a2f-4c54-8a7b-fd8ed76610f7","Type":"ContainerDied","Data":"6d3706d33cf3fd93154e42d96c02c213d6f60c1407609eb6f765aa1dff2d2066"}
Jul 26 17:00:02.160716 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.160696    2570 generic.go:356] "Generic (PLEG): container finished" podID="a4e81ed2-1773-4760-989a-ff81685d0426" containerID="a4255fa8c54f62c0392985a385c0388fad95cc906bf0d9796e4aa2fa83ac55b3" exitCode=0
Jul 26 17:00:02.160816 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.160775    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm" event={"ID":"a4e81ed2-1773-4760-989a-ff81685d0426","Type":"ContainerDied","Data":"a4255fa8c54f62c0392985a385c0388fad95cc906bf0d9796e4aa2fa83ac55b3"}
Jul 26 17:00:02.162287 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.162258    2570 generic.go:356] "Generic (PLEG): container finished" podID="f285656e-c485-4685-8c9d-62c097c187fe" containerID="dfecdd51195af518544069263d432fe08bde347e7bb48a496562adf7f7be4f35" exitCode=0
Jul 26 17:00:02.162371 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:02.162289    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf" event={"ID":"f285656e-c485-4685-8c9d-62c097c187fe","Type":"ContainerDied","Data":"dfecdd51195af518544069263d432fe08bde347e7bb48a496562adf7f7be4f35"}
Jul 26 17:00:03.358317 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:03.358290    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d"
Jul 26 17:00:03.363828 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:03.363754    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm"
Jul 26 17:00:03.400057 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:03.399966    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv"
Jul 26 17:00:03.404556 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:03.404537    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29"
Jul 26 17:00:03.408054 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:03.408039    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf"
Jul 26 17:00:04.173500 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.173468    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29"
Jul 26 17:00:04.173500 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.173482    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29" event={"ID":"677c510a-71d1-4d79-8f5d-0ec0b5c82184","Type":"ContainerDied","Data":"9c09d55a41e8080a7242f95609ea0d8f480e05f19e69c69d874c564ae88b06c6"}
Jul 26 17:00:04.173760 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.173526    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c09d55a41e8080a7242f95609ea0d8f480e05f19e69c69d874c564ae88b06c6"
Jul 26 17:00:04.177653 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.177632    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d"
Jul 26 17:00:04.177772 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.177664    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d" event={"ID":"e6b7f7f4-6a2f-4c54-8a7b-fd8ed76610f7","Type":"ContainerDied","Data":"60620c794bdfa2698d3fd423c15d631cd4dadfa1001c8dd962a7bb8daa685f26"}
Jul 26 17:00:04.177772 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.177704    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60620c794bdfa2698d3fd423c15d631cd4dadfa1001c8dd962a7bb8daa685f26"
Jul 26 17:00:04.179068 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.179033    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm" event={"ID":"a4e81ed2-1773-4760-989a-ff81685d0426","Type":"ContainerDied","Data":"dddf8e783b31e9106b9964070a25718c1e0a0a954ac3565f1cd5362e509d6028"}
Jul 26 17:00:04.179205 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.179077    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dddf8e783b31e9106b9964070a25718c1e0a0a954ac3565f1cd5362e509d6028"
Jul 26 17:00:04.179205 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.179042    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm"
Jul 26 17:00:04.180473 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.180454    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf"
Jul 26 17:00:04.180563 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.180508    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751420-hgvnf" event={"ID":"f285656e-c485-4685-8c9d-62c097c187fe","Type":"ContainerDied","Data":"73edefca23037211b127fc5419f910ff44e52f7dac93825a92d962de1b90c0f4"}
Jul 26 17:00:04.180563 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.180544    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73edefca23037211b127fc5419f910ff44e52f7dac93825a92d962de1b90c0f4"
Jul 26 17:00:04.181926 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.181899    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv" event={"ID":"0b8e02d9-65ec-4807-850d-786ddeb247e5","Type":"ContainerDied","Data":"67678d58bfb81d720e8779174892140af5db4c8310c1f819f465ef623a3979df"}
Jul 26 17:00:04.182034 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.181924    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv"
Jul 26 17:00:04.182034 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:04.181943    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67678d58bfb81d720e8779174892140af5db4c8310c1f819f465ef623a3979df"
Jul 26 17:00:11.334050 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:11.334018    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr_2747ef12-136d-4b51-8c1f-1d876b0d7be4/payload-processing/0.log"
Jul 26 17:00:14.449071 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449041    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"]
Jul 26 17:00:14.449487 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449470    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f285656e-c485-4685-8c9d-62c097c187fe" containerName="cleanup"
Jul 26 17:00:14.449487 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449488    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f285656e-c485-4685-8c9d-62c097c187fe" containerName="cleanup"
Jul 26 17:00:14.449603 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449501    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b8e02d9-65ec-4807-850d-786ddeb247e5" containerName="cleanup"
Jul 26 17:00:14.449603 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449509    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b8e02d9-65ec-4807-850d-786ddeb247e5" containerName="cleanup"
Jul 26 17:00:14.449603 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449525    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="677c510a-71d1-4d79-8f5d-0ec0b5c82184" containerName="cleanup"
Jul 26 17:00:14.449603 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449534    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="677c510a-71d1-4d79-8f5d-0ec0b5c82184" containerName="cleanup"
Jul 26 17:00:14.449603 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449551    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b7f7f4-6a2f-4c54-8a7b-fd8ed76610f7" containerName="cleanup"
Jul 26 17:00:14.449603 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449560    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b7f7f4-6a2f-4c54-8a7b-fd8ed76610f7" containerName="cleanup"
Jul 26 17:00:14.449603 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449575    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4e81ed2-1773-4760-989a-ff81685d0426" containerName="cleanup"
Jul 26 17:00:14.449603 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449582    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4e81ed2-1773-4760-989a-ff81685d0426" containerName="cleanup"
Jul 26 17:00:14.449866 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449660    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f285656e-c485-4685-8c9d-62c097c187fe" containerName="cleanup"
Jul 26 17:00:14.449866 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449670    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4e81ed2-1773-4760-989a-ff81685d0426" containerName="cleanup"
Jul 26 17:00:14.449866 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449678    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b7f7f4-6a2f-4c54-8a7b-fd8ed76610f7" containerName="cleanup"
Jul 26 17:00:14.449866 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449686    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b8e02d9-65ec-4807-850d-786ddeb247e5" containerName="cleanup"
Jul 26 17:00:14.449866 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.449695    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="677c510a-71d1-4d79-8f5d-0ec0b5c82184" containerName="cleanup"
Jul 26 17:00:14.456981 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.456960    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.461524 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.461500    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-50e630\"/\"kube-root-ca.crt\""
Jul 26 17:00:14.461795 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.461775    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-50e630\"/\"default-dockercfg-dcz86\""
Jul 26 17:00:14.462074 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.462041    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-50e630\"/\"openshift-service-ca.crt\""
Jul 26 17:00:14.462202 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.462151    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-50e630\"/\"ipp-route-50e630-kserve-self-signed-certs\""
Jul 26 17:00:14.463964 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.463944    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"]
Jul 26 17:00:14.491216 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.491173    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tls-certs\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.491363 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.491266    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-model-cache\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.491363 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.491313    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-home\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.491363 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.491334    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tmp-dir\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.491507 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.491368    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-dshm\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.491507 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.491392    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pldcv\" (UniqueName: \"kubernetes.io/projected/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-kube-api-access-pldcv\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.592214 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.592165    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-home\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.592214 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.592212    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tmp-dir\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.592447 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.592234    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-dshm\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.592447 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.592254    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pldcv\" (UniqueName: \"kubernetes.io/projected/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-kube-api-access-pldcv\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.592447 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.592308    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tls-certs\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.592447 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.592365    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-model-cache\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.592672 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.592649    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-home\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.592723 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.592678    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tmp-dir\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.592807 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.592785    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-model-cache\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.594610 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.594594    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-dshm\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.594966 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.594945    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tls-certs\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.600816 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.600790    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pldcv\" (UniqueName: \"kubernetes.io/projected/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-kube-api-access-pldcv\") pod \"ipp-route-50e630-kserve-775979c467-hqmgh\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") " pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.768857 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.768772    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:14.897674 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:14.897649    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"]
Jul 26 17:00:14.899816 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:00:14.899791    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac865c22_7af2_4f1a_8d1c_ece22d3cd303.slice/crio-b2539f6a7b5953d3288f455c5440c1383284e39b83f72db69edead1d8339ac54 WatchSource:0}: Error finding container b2539f6a7b5953d3288f455c5440c1383284e39b83f72db69edead1d8339ac54: Status 404 returned error can't find the container with id b2539f6a7b5953d3288f455c5440c1383284e39b83f72db69edead1d8339ac54
Jul 26 17:00:15.234166 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:15.234133    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh" event={"ID":"ac865c22-7af2-4f1a-8d1c-ece22d3cd303","Type":"ContainerStarted","Data":"9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c"}
Jul 26 17:00:15.234166 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:15.234172    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh" event={"ID":"ac865c22-7af2-4f1a-8d1c-ece22d3cd303","Type":"ContainerStarted","Data":"b2539f6a7b5953d3288f455c5440c1383284e39b83f72db69edead1d8339ac54"}
Jul 26 17:00:15.253425 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:15.253378    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh" podStartSLOduration=1.253365212 podStartE2EDuration="1.253365212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:00:15.251183491 +0000 UTC m=+3111.434267780" watchObservedRunningTime="2026-07-26 17:00:15.253365212 +0000 UTC m=+3111.436449499"
Jul 26 17:00:18.310884 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:18.310850    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 17:00:24.769580 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:24.769545    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:24.771882 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:24.769804    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:24.782033 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:24.782008    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:25.302401 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:25.302373    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:00:45.912743 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:45.912702    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 17:00:56.306014 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:00:56.305940    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr_2747ef12-136d-4b51-8c1f-1d876b0d7be4/payload-processing/0.log"
Jul 26 17:01:02.036483 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.036448    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"]
Jul 26 17:01:02.040050 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.040034    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.042939 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.042916    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-54f5f8-serving-cert\""
Jul 26 17:01:02.049480 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.049456    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"]
Jul 26 17:01:02.129354 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.129318    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fd9dd9-ac16-4f73-9525-a3b53fda2130-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.129540 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.129372    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fd9dd9-ac16-4f73-9525-a3b53fda2130-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.129540 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.129436    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz22v\" (UniqueName: \"kubernetes.io/projected/46fd9dd9-ac16-4f73-9525-a3b53fda2130-kube-api-access-qz22v\") pod \"maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.230916 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.230878    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fd9dd9-ac16-4f73-9525-a3b53fda2130-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.231112 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.230930    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fd9dd9-ac16-4f73-9525-a3b53fda2130-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.231112 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.230971    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz22v\" (UniqueName: \"kubernetes.io/projected/46fd9dd9-ac16-4f73-9525-a3b53fda2130-kube-api-access-qz22v\") pod \"maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.231112 ip-10-0-135-132 kubenswrapper[2570]: E0726 17:01:02.231032    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-serving-cert: secret "maas-api-e2e-ipp-cleanup-54f5f8-serving-cert" not found
Jul 26 17:01:02.231112 ip-10-0-135-132 kubenswrapper[2570]: E0726 17:01:02.231101    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46fd9dd9-ac16-4f73-9525-a3b53fda2130-maas-api-tls podName:46fd9dd9-ac16-4f73-9525-a3b53fda2130 nodeName:}" failed. No retries permitted until 2026-07-26 17:01:02.731084564 +0000 UTC m=+3158.914168828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/46fd9dd9-ac16-4f73-9525-a3b53fda2130-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt" (UID: "46fd9dd9-ac16-4f73-9525-a3b53fda2130") : secret "maas-api-e2e-ipp-cleanup-54f5f8-serving-cert" not found
Jul 26 17:01:02.231700 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.231679    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fd9dd9-ac16-4f73-9525-a3b53fda2130-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.239927 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.239900    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz22v\" (UniqueName: \"kubernetes.io/projected/46fd9dd9-ac16-4f73-9525-a3b53fda2130-kube-api-access-qz22v\") pod \"maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.705092 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.705054    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"]
Jul 26 17:01:02.712890 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.712870    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:02.715861 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.715839    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-54f5f8\""
Jul 26 17:01:02.716032 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.716008    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-54f5f8-dockercfg-ktxts\""
Jul 26 17:01:02.732987 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.732958    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"]
Jul 26 17:01:02.735118 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.735096    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc\" (UID: \"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:02.735261 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.735127    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ftnm\" (UniqueName: \"kubernetes.io/projected/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-kube-api-access-2ftnm\") pod \"payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc\" (UID: \"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:02.735261 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.735173    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fd9dd9-ac16-4f73-9525-a3b53fda2130-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.737554 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.737534    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fd9dd9-ac16-4f73-9525-a3b53fda2130-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.752016 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.751988    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"]
Jul 26 17:01:02.761139 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.761118    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:02.794031 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.793998    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"]
Jul 26 17:01:02.836229 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.836165    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc\" (UID: \"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:02.836229 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.836231    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ftnm\" (UniqueName: \"kubernetes.io/projected/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-kube-api-access-2ftnm\") pod \"payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc\" (UID: \"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:02.836436 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.836274    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5wpl\" (UniqueName: \"kubernetes.io/projected/205ce4b6-9c72-4099-bfea-70715ea9f128-kube-api-access-d5wpl\") pod \"payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj\" (UID: \"205ce4b6-9c72-4099-bfea-70715ea9f128\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:02.836436 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.836344    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205ce4b6-9c72-4099-bfea-70715ea9f128-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj\" (UID: \"205ce4b6-9c72-4099-bfea-70715ea9f128\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:02.836914 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.836895    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc\" (UID: \"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:02.850851 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.850827    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ftnm\" (UniqueName: \"kubernetes.io/projected/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-kube-api-access-2ftnm\") pod \"payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc\" (UID: \"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:02.937754 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.937713    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5wpl\" (UniqueName: \"kubernetes.io/projected/205ce4b6-9c72-4099-bfea-70715ea9f128-kube-api-access-d5wpl\") pod \"payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj\" (UID: \"205ce4b6-9c72-4099-bfea-70715ea9f128\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:02.937964 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.937802    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205ce4b6-9c72-4099-bfea-70715ea9f128-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj\" (UID: \"205ce4b6-9c72-4099-bfea-70715ea9f128\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:02.938520 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.938501    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205ce4b6-9c72-4099-bfea-70715ea9f128-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj\" (UID: \"205ce4b6-9c72-4099-bfea-70715ea9f128\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:02.951116 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.951083    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:02.956139 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:02.956115    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5wpl\" (UniqueName: \"kubernetes.io/projected/205ce4b6-9c72-4099-bfea-70715ea9f128-kube-api-access-d5wpl\") pod \"payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj\" (UID: \"205ce4b6-9c72-4099-bfea-70715ea9f128\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:03.023371 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.023339    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:03.071641 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.071560    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:03.175105 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.174901    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"]
Jul 26 17:01:03.177423 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:01:03.177398    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4806ecb_e6bd_4151_9d6d_c67cc6d6e316.slice/crio-e09c7adf30be4ca3132be16e322c32e42aaddce761c5bd8d3a84ef5b6c46f15d WatchSource:0}: Error finding container e09c7adf30be4ca3132be16e322c32e42aaddce761c5bd8d3a84ef5b6c46f15d: Status 404 returned error can't find the container with id e09c7adf30be4ca3132be16e322c32e42aaddce761c5bd8d3a84ef5b6c46f15d
Jul 26 17:01:03.217865 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.217844    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"]
Jul 26 17:01:03.220174 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:01:03.220147    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205ce4b6_9c72_4099_bfea_70715ea9f128.slice/crio-2b25bf92b6211c4998f55bf040ec7987541cfb65ae5ba5a46e04fd5591720a29 WatchSource:0}: Error finding container 2b25bf92b6211c4998f55bf040ec7987541cfb65ae5ba5a46e04fd5591720a29: Status 404 returned error can't find the container with id 2b25bf92b6211c4998f55bf040ec7987541cfb65ae5ba5a46e04fd5591720a29
Jul 26 17:01:03.293772 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.293745    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"]
Jul 26 17:01:03.295562 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:01:03.295538    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46fd9dd9_ac16_4f73_9525_a3b53fda2130.slice/crio-9bf9b665a882d30e6ed0116b37d79a1e5ae9156028ef43251a7b1d77572d61c8 WatchSource:0}: Error finding container 9bf9b665a882d30e6ed0116b37d79a1e5ae9156028ef43251a7b1d77572d61c8: Status 404 returned error can't find the container with id 9bf9b665a882d30e6ed0116b37d79a1e5ae9156028ef43251a7b1d77572d61c8
Jul 26 17:01:03.470809 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.470717    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj" event={"ID":"205ce4b6-9c72-4099-bfea-70715ea9f128","Type":"ContainerStarted","Data":"972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5"}
Jul 26 17:01:03.470809 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.470769    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj" event={"ID":"205ce4b6-9c72-4099-bfea-70715ea9f128","Type":"ContainerStarted","Data":"2b25bf92b6211c4998f55bf040ec7987541cfb65ae5ba5a46e04fd5591720a29"}
Jul 26 17:01:03.471057 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.470823    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:03.471986 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.471966    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt" event={"ID":"46fd9dd9-ac16-4f73-9525-a3b53fda2130","Type":"ContainerStarted","Data":"9bf9b665a882d30e6ed0116b37d79a1e5ae9156028ef43251a7b1d77572d61c8"}
Jul 26 17:01:03.473387 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.473369    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc" event={"ID":"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316","Type":"ContainerStarted","Data":"dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad"}
Jul 26 17:01:03.473460 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.473401    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc" event={"ID":"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316","Type":"ContainerStarted","Data":"e09c7adf30be4ca3132be16e322c32e42aaddce761c5bd8d3a84ef5b6c46f15d"}
Jul 26 17:01:03.473460 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.473421    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:03.488344 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.488296    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj" podStartSLOduration=1.488281752 podStartE2EDuration="1.488281752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:01:03.487516981 +0000 UTC m=+3159.670601269" watchObservedRunningTime="2026-07-26 17:01:03.488281752 +0000 UTC m=+3159.671366039"
Jul 26 17:01:03.509281 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:03.509223    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc" podStartSLOduration=1.509205479 podStartE2EDuration="1.509205479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:01:03.505776335 +0000 UTC m=+3159.688860642" watchObservedRunningTime="2026-07-26 17:01:03.509205479 +0000 UTC m=+3159.692289762"
Jul 26 17:01:04.479609 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:04.479570    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt" event={"ID":"46fd9dd9-ac16-4f73-9525-a3b53fda2130","Type":"ContainerStarted","Data":"f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a"}
Jul 26 17:01:04.499391 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:04.499324    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt" podStartSLOduration=2.019053131 podStartE2EDuration="2.499304261s" totalImagesPullingTime="480.25113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:01:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:01:04.496988291 +0000 UTC m=+3160.680072579" watchObservedRunningTime="2026-07-26 17:01:04.499304261 +0000 UTC m=+3160.682388550"
Jul 26 17:01:05.485224 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:05.485182    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:05.849425 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:05.849336    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 17:01:11.496227 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:11.496174    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:14.481654 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:14.481623    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:14.482131 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:14.481881    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:28.864470 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:28.864425    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"]
Jul 26 17:01:28.864936 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:28.864684    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt" podUID="46fd9dd9-ac16-4f73-9525-a3b53fda2130" containerName="maas-api" containerID="cri-o://f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a" gracePeriod=30
Jul 26 17:01:28.919102 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:28.919066    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"]
Jul 26 17:01:28.919394 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:28.919334    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc" podUID="e4806ecb-e6bd-4151-9d6d-c67cc6d6e316" containerName="payload-processing" containerID="cri-o://dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad" gracePeriod=30
Jul 26 17:01:28.925783 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:28.925754    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"]
Jul 26 17:01:28.926159 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:28.925985    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj" podUID="205ce4b6-9c72-4099-bfea-70715ea9f128" containerName="payload-pre-processing" containerID="cri-o://972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5" gracePeriod=30
Jul 26 17:01:29.179240 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.179208    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:29.215564 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.215540    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:29.221415 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.221388    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:29.283135 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.283103    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205ce4b6-9c72-4099-bfea-70715ea9f128-plugins-config-volume\") pod \"205ce4b6-9c72-4099-bfea-70715ea9f128\" (UID: \"205ce4b6-9c72-4099-bfea-70715ea9f128\") "
Jul 26 17:01:29.283330 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.283309    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5wpl\" (UniqueName: \"kubernetes.io/projected/205ce4b6-9c72-4099-bfea-70715ea9f128-kube-api-access-d5wpl\") pod \"205ce4b6-9c72-4099-bfea-70715ea9f128\" (UID: \"205ce4b6-9c72-4099-bfea-70715ea9f128\") "
Jul 26 17:01:29.283441 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.283421    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/205ce4b6-9c72-4099-bfea-70715ea9f128-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "205ce4b6-9c72-4099-bfea-70715ea9f128" (UID: "205ce4b6-9c72-4099-bfea-70715ea9f128"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:01:29.283645 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.283628    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205ce4b6-9c72-4099-bfea-70715ea9f128-plugins-config-volume\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:01:29.285464 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.285438    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/205ce4b6-9c72-4099-bfea-70715ea9f128-kube-api-access-d5wpl" (OuterVolumeSpecName: "kube-api-access-d5wpl") pod "205ce4b6-9c72-4099-bfea-70715ea9f128" (UID: "205ce4b6-9c72-4099-bfea-70715ea9f128"). InnerVolumeSpecName "kube-api-access-d5wpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:01:29.383939 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.383867    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fd9dd9-ac16-4f73-9525-a3b53fda2130-maas-api-tls\") pod \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") "
Jul 26 17:01:29.383939 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.383908    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fd9dd9-ac16-4f73-9525-a3b53fda2130-openshift-service-ca\") pod \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") "
Jul 26 17:01:29.383939 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.383931    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-plugins-config-volume\") pod \"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316\" (UID: \"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316\") "
Jul 26 17:01:29.384231 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.383949    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ftnm\" (UniqueName: \"kubernetes.io/projected/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-kube-api-access-2ftnm\") pod \"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316\" (UID: \"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316\") "
Jul 26 17:01:29.384231 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.384002    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz22v\" (UniqueName: \"kubernetes.io/projected/46fd9dd9-ac16-4f73-9525-a3b53fda2130-kube-api-access-qz22v\") pod \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\" (UID: \"46fd9dd9-ac16-4f73-9525-a3b53fda2130\") "
Jul 26 17:01:29.384351 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.384313    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46fd9dd9-ac16-4f73-9525-a3b53fda2130-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46fd9dd9-ac16-4f73-9525-a3b53fda2130" (UID: "46fd9dd9-ac16-4f73-9525-a3b53fda2130"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:01:29.384351 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.384329    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e4806ecb-e6bd-4151-9d6d-c67cc6d6e316" (UID: "e4806ecb-e6bd-4151-9d6d-c67cc6d6e316"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:01:29.384478 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.384460    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fd9dd9-ac16-4f73-9525-a3b53fda2130-openshift-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:01:29.384531 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.384483    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-plugins-config-volume\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:01:29.384531 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.384497    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5wpl\" (UniqueName: \"kubernetes.io/projected/205ce4b6-9c72-4099-bfea-70715ea9f128-kube-api-access-d5wpl\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:01:29.386182 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.386155    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-kube-api-access-2ftnm" (OuterVolumeSpecName: "kube-api-access-2ftnm") pod "e4806ecb-e6bd-4151-9d6d-c67cc6d6e316" (UID: "e4806ecb-e6bd-4151-9d6d-c67cc6d6e316"). InnerVolumeSpecName "kube-api-access-2ftnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:01:29.386498 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.386477    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46fd9dd9-ac16-4f73-9525-a3b53fda2130-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46fd9dd9-ac16-4f73-9525-a3b53fda2130" (UID: "46fd9dd9-ac16-4f73-9525-a3b53fda2130"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:01:29.386552 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.386529    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46fd9dd9-ac16-4f73-9525-a3b53fda2130-kube-api-access-qz22v" (OuterVolumeSpecName: "kube-api-access-qz22v") pod "46fd9dd9-ac16-4f73-9525-a3b53fda2130" (UID: "46fd9dd9-ac16-4f73-9525-a3b53fda2130"). InnerVolumeSpecName "kube-api-access-qz22v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:01:29.485947 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.485910    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz22v\" (UniqueName: \"kubernetes.io/projected/46fd9dd9-ac16-4f73-9525-a3b53fda2130-kube-api-access-qz22v\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:01:29.485947 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.485946    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fd9dd9-ac16-4f73-9525-a3b53fda2130-maas-api-tls\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:01:29.486149 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.485956    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ftnm\" (UniqueName: \"kubernetes.io/projected/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316-kube-api-access-2ftnm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:01:29.610640 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.610603    2570 generic.go:356] "Generic (PLEG): container finished" podID="46fd9dd9-ac16-4f73-9525-a3b53fda2130" containerID="f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a" exitCode=0
Jul 26 17:01:29.610840 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.610668    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"
Jul 26 17:01:29.610840 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.610683    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt" event={"ID":"46fd9dd9-ac16-4f73-9525-a3b53fda2130","Type":"ContainerDied","Data":"f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a"}
Jul 26 17:01:29.610840 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.610787    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt" event={"ID":"46fd9dd9-ac16-4f73-9525-a3b53fda2130","Type":"ContainerDied","Data":"9bf9b665a882d30e6ed0116b37d79a1e5ae9156028ef43251a7b1d77572d61c8"}
Jul 26 17:01:29.610840 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.610827    2570 scope.go:117] "RemoveContainer" containerID="f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a"
Jul 26 17:01:29.612121 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.612097    2570 generic.go:356] "Generic (PLEG): container finished" podID="e4806ecb-e6bd-4151-9d6d-c67cc6d6e316" containerID="dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad" exitCode=0
Jul 26 17:01:29.612274 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.612169    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"
Jul 26 17:01:29.612274 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.612179    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc" event={"ID":"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316","Type":"ContainerDied","Data":"dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad"}
Jul 26 17:01:29.612274 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.612254    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc" event={"ID":"e4806ecb-e6bd-4151-9d6d-c67cc6d6e316","Type":"ContainerDied","Data":"e09c7adf30be4ca3132be16e322c32e42aaddce761c5bd8d3a84ef5b6c46f15d"}
Jul 26 17:01:29.613849 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.613821    2570 generic.go:356] "Generic (PLEG): container finished" podID="205ce4b6-9c72-4099-bfea-70715ea9f128" containerID="972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5" exitCode=0
Jul 26 17:01:29.613953 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.613868    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"
Jul 26 17:01:29.613953 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.613875    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj" event={"ID":"205ce4b6-9c72-4099-bfea-70715ea9f128","Type":"ContainerDied","Data":"972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5"}
Jul 26 17:01:29.613953 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.613916    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj" event={"ID":"205ce4b6-9c72-4099-bfea-70715ea9f128","Type":"ContainerDied","Data":"2b25bf92b6211c4998f55bf040ec7987541cfb65ae5ba5a46e04fd5591720a29"}
Jul 26 17:01:29.621424 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.621403    2570 scope.go:117] "RemoveContainer" containerID="f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a"
Jul 26 17:01:29.621703 ip-10-0-135-132 kubenswrapper[2570]: E0726 17:01:29.621680    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a\": container with ID starting with f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a not found: ID does not exist" containerID="f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a"
Jul 26 17:01:29.621808 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.621713    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a"} err="failed to get container status \"f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a\": rpc error: code = NotFound desc = could not find container \"f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a\": container with ID starting with f220b2979cf5f55aabe8b5cab3eb555e4947153d33372546d73845b8b868a90a not found: ID does not exist"
Jul 26 17:01:29.621808 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.621735    2570 scope.go:117] "RemoveContainer" containerID="dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad"
Jul 26 17:01:29.629703 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.629686    2570 scope.go:117] "RemoveContainer" containerID="dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad"
Jul 26 17:01:29.629945 ip-10-0-135-132 kubenswrapper[2570]: E0726 17:01:29.629929    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad\": container with ID starting with dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad not found: ID does not exist" containerID="dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad"
Jul 26 17:01:29.629992 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.629951    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad"} err="failed to get container status \"dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad\": rpc error: code = NotFound desc = could not find container \"dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad\": container with ID starting with dd62a6cb1934aff1bdfe37206ca2256f14780579ab4b6f76cb8ae1dc0b8507ad not found: ID does not exist"
Jul 26 17:01:29.629992 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.629966    2570 scope.go:117] "RemoveContainer" containerID="972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5"
Jul 26 17:01:29.635314 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.635253    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"]
Jul 26 17:01:29.637093 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.637070    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-54f5f8-67d84575b-26gzt"]
Jul 26 17:01:29.637963 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.637945    2570 scope.go:117] "RemoveContainer" containerID="972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5"
Jul 26 17:01:29.638239 ip-10-0-135-132 kubenswrapper[2570]: E0726 17:01:29.638216    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5\": container with ID starting with 972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5 not found: ID does not exist" containerID="972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5"
Jul 26 17:01:29.638327 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.638247    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5"} err="failed to get container status \"972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5\": rpc error: code = NotFound desc = could not find container \"972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5\": container with ID starting with 972dda0e80b498479bdace4c085847ecfe9ced70b5ad1d8fe2f0eb408eda57f5 not found: ID does not exist"
Jul 26 17:01:29.647108 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.647082    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"]
Jul 26 17:01:29.653320 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.653299    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-54f5f8-57dcd9cf64-dcsjc"]
Jul 26 17:01:29.664518 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.664489    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"]
Jul 26 17:01:29.670094 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:29.670068    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-54f5f8-597bbd7996-zkcfj"]
Jul 26 17:01:30.302536 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:30.302502    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="205ce4b6-9c72-4099-bfea-70715ea9f128" path="/var/lib/kubelet/pods/205ce4b6-9c72-4099-bfea-70715ea9f128/volumes"
Jul 26 17:01:30.302926 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:30.302906    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46fd9dd9-ac16-4f73-9525-a3b53fda2130" path="/var/lib/kubelet/pods/46fd9dd9-ac16-4f73-9525-a3b53fda2130/volumes"
Jul 26 17:01:30.303246 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:30.303230    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4806ecb-e6bd-4151-9d6d-c67cc6d6e316" path="/var/lib/kubelet/pods/e4806ecb-e6bd-4151-9d6d-c67cc6d6e316/volumes"
Jul 26 17:01:37.721825 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:37.721788    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 17:01:40.723456 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.723420    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"]
Jul 26 17:01:40.723849 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.723775    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4806ecb-e6bd-4151-9d6d-c67cc6d6e316" containerName="payload-processing"
Jul 26 17:01:40.723849 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.723786    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4806ecb-e6bd-4151-9d6d-c67cc6d6e316" containerName="payload-processing"
Jul 26 17:01:40.723849 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.723813    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="205ce4b6-9c72-4099-bfea-70715ea9f128" containerName="payload-pre-processing"
Jul 26 17:01:40.723849 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.723819    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="205ce4b6-9c72-4099-bfea-70715ea9f128" containerName="payload-pre-processing"
Jul 26 17:01:40.723849 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.723836    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46fd9dd9-ac16-4f73-9525-a3b53fda2130" containerName="maas-api"
Jul 26 17:01:40.723849 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.723843    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="46fd9dd9-ac16-4f73-9525-a3b53fda2130" containerName="maas-api"
Jul 26 17:01:40.724068 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.723910    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="205ce4b6-9c72-4099-bfea-70715ea9f128" containerName="payload-pre-processing"
Jul 26 17:01:40.724068 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.723920    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="46fd9dd9-ac16-4f73-9525-a3b53fda2130" containerName="maas-api"
Jul 26 17:01:40.724068 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.723926    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4806ecb-e6bd-4151-9d6d-c67cc6d6e316" containerName="payload-processing"
Jul 26 17:01:40.726817 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.726793    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"
Jul 26 17:01:40.729496 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.729473    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2xlcn\""
Jul 26 17:01:40.734764 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.734737    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"]
Jul 26 17:01:40.774095 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.774064    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bb11d8-b453-49ad-8308-f47e1f2c9ccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5\" (UID: \"66bb11d8-b453-49ad-8308-f47e1f2c9ccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"
Jul 26 17:01:40.875098 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.875066    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bb11d8-b453-49ad-8308-f47e1f2c9ccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5\" (UID: \"66bb11d8-b453-49ad-8308-f47e1f2c9ccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"
Jul 26 17:01:40.875687 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:40.875666    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bb11d8-b453-49ad-8308-f47e1f2c9ccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5\" (UID: \"66bb11d8-b453-49ad-8308-f47e1f2c9ccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"
Jul 26 17:01:41.038579 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:41.038498    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"
Jul 26 17:01:41.173151 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:41.173125    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"]
Jul 26 17:01:41.174945 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:01:41.174920    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66bb11d8_b453_49ad_8308_f47e1f2c9ccc.slice/crio-e8d05460799033a17f31b2203009236886dd152ed32248b1cf1be85174e84570 WatchSource:0}: Error finding container e8d05460799033a17f31b2203009236886dd152ed32248b1cf1be85174e84570: Status 404 returned error can't find the container with id e8d05460799033a17f31b2203009236886dd152ed32248b1cf1be85174e84570
Jul 26 17:01:41.676440 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:41.676366    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5" event={"ID":"66bb11d8-b453-49ad-8308-f47e1f2c9ccc","Type":"ContainerStarted","Data":"2b4b4aa95f115aa8eb4f3a032d7a6df648f9665f2ace17cafcc9a4ac5ad262a4"}
Jul 26 17:01:41.676440 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:41.676416    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5" event={"ID":"66bb11d8-b453-49ad-8308-f47e1f2c9ccc","Type":"ContainerStarted","Data":"e8d05460799033a17f31b2203009236886dd152ed32248b1cf1be85174e84570"}
Jul 26 17:01:41.692712 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:41.692659    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5" podStartSLOduration=1.692643468 podStartE2EDuration="1.692643468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:01:41.691411812 +0000 UTC m=+3197.874496100" watchObservedRunningTime="2026-07-26 17:01:41.692643468 +0000 UTC m=+3197.875727754"
Jul 26 17:01:42.682605 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:42.682571    2570 generic.go:356] "Generic (PLEG): container finished" podID="66bb11d8-b453-49ad-8308-f47e1f2c9ccc" containerID="2b4b4aa95f115aa8eb4f3a032d7a6df648f9665f2ace17cafcc9a4ac5ad262a4" exitCode=0
Jul 26 17:01:42.682983 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:42.682649    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5" event={"ID":"66bb11d8-b453-49ad-8308-f47e1f2c9ccc","Type":"ContainerDied","Data":"2b4b4aa95f115aa8eb4f3a032d7a6df648f9665f2ace17cafcc9a4ac5ad262a4"}
Jul 26 17:01:43.822463 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:43.822440    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"
Jul 26 17:01:43.896447 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:43.896414    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bb11d8-b453-49ad-8308-f47e1f2c9ccc-maas-api-service-ca\") pod \"66bb11d8-b453-49ad-8308-f47e1f2c9ccc\" (UID: \"66bb11d8-b453-49ad-8308-f47e1f2c9ccc\") "
Jul 26 17:01:43.896751 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:43.896729    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66bb11d8-b453-49ad-8308-f47e1f2c9ccc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66bb11d8-b453-49ad-8308-f47e1f2c9ccc" (UID: "66bb11d8-b453-49ad-8308-f47e1f2c9ccc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:01:43.997205 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:43.997156    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bb11d8-b453-49ad-8308-f47e1f2c9ccc-maas-api-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:01:44.695136 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:44.695115    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"
Jul 26 17:01:44.695334 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:44.695110    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5" event={"ID":"66bb11d8-b453-49ad-8308-f47e1f2c9ccc","Type":"ContainerDied","Data":"e8d05460799033a17f31b2203009236886dd152ed32248b1cf1be85174e84570"}
Jul 26 17:01:44.695334 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:44.695283    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8d05460799033a17f31b2203009236886dd152ed32248b1cf1be85174e84570"
Jul 26 17:01:57.321934 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:01:57.321899    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 17:02:01.192948 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:01.192915    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv"]
Jul 26 17:02:01.195854 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:01.195830    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-50e630-29751420-6v8xv"]
Jul 26 17:02:02.300129 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:02.300095    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b8e02d9-65ec-4807-850d-786ddeb247e5" path="/var/lib/kubelet/pods/0b8e02d9-65ec-4807-850d-786ddeb247e5/volumes"
Jul 26 17:02:05.835663 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:05.835626    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"]
Jul 26 17:02:05.838652 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:05.838624    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-50e630-ggwz5"]
Jul 26 17:02:06.301388 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:06.301352    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66bb11d8-b453-49ad-8308-f47e1f2c9ccc" path="/var/lib/kubelet/pods/66bb11d8-b453-49ad-8308-f47e1f2c9ccc/volumes"
Jul 26 17:02:11.204904 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:11.204865    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"]
Jul 26 17:02:13.022080 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:13.022044    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 17:02:26.224653 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.224585    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh" podUID="ac865c22-7af2-4f1a-8d1c-ece22d3cd303" containerName="main" containerID="cri-o://9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c" gracePeriod=45
Jul 26 17:02:26.468853 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.468830    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:02:26.585882 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.585800    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-model-cache\") pod \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") "
Jul 26 17:02:26.586011 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.585911    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tmp-dir\") pod \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") "
Jul 26 17:02:26.586011 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.585934    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-home\") pod \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") "
Jul 26 17:02:26.586011 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.585948    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-dshm\") pod \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") "
Jul 26 17:02:26.586011 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.585966    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pldcv\" (UniqueName: \"kubernetes.io/projected/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-kube-api-access-pldcv\") pod \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") "
Jul 26 17:02:26.586011 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.586000    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tls-certs\") pod \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\" (UID: \"ac865c22-7af2-4f1a-8d1c-ece22d3cd303\") "
Jul 26 17:02:26.586288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.586096    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-model-cache" (OuterVolumeSpecName: "model-cache") pod "ac865c22-7af2-4f1a-8d1c-ece22d3cd303" (UID: "ac865c22-7af2-4f1a-8d1c-ece22d3cd303"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:02:26.586288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.586241    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-home" (OuterVolumeSpecName: "home") pod "ac865c22-7af2-4f1a-8d1c-ece22d3cd303" (UID: "ac865c22-7af2-4f1a-8d1c-ece22d3cd303"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:02:26.586288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.586264    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ac865c22-7af2-4f1a-8d1c-ece22d3cd303" (UID: "ac865c22-7af2-4f1a-8d1c-ece22d3cd303"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:02:26.586394 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.586366    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:02:26.586394 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.586383    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:02:26.586489 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.586394    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:02:26.588157 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.588131    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ac865c22-7af2-4f1a-8d1c-ece22d3cd303" (UID: "ac865c22-7af2-4f1a-8d1c-ece22d3cd303"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:02:26.588268 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.588149    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-dshm" (OuterVolumeSpecName: "dshm") pod "ac865c22-7af2-4f1a-8d1c-ece22d3cd303" (UID: "ac865c22-7af2-4f1a-8d1c-ece22d3cd303"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:02:26.588333 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.588289    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-kube-api-access-pldcv" (OuterVolumeSpecName: "kube-api-access-pldcv") pod "ac865c22-7af2-4f1a-8d1c-ece22d3cd303" (UID: "ac865c22-7af2-4f1a-8d1c-ece22d3cd303"). InnerVolumeSpecName "kube-api-access-pldcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:02:26.686951 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.686920    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:02:26.686951 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.686946    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pldcv\" (UniqueName: \"kubernetes.io/projected/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-kube-api-access-pldcv\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:02:26.686951 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.686957    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac865c22-7af2-4f1a-8d1c-ece22d3cd303-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:02:26.895789 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.895681    2570 generic.go:356] "Generic (PLEG): container finished" podID="ac865c22-7af2-4f1a-8d1c-ece22d3cd303" containerID="9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c" exitCode=0
Jul 26 17:02:26.895789 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.895747    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"
Jul 26 17:02:26.895789 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.895772    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh" event={"ID":"ac865c22-7af2-4f1a-8d1c-ece22d3cd303","Type":"ContainerDied","Data":"9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c"}
Jul 26 17:02:26.896080 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.895820    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh" event={"ID":"ac865c22-7af2-4f1a-8d1c-ece22d3cd303","Type":"ContainerDied","Data":"b2539f6a7b5953d3288f455c5440c1383284e39b83f72db69edead1d8339ac54"}
Jul 26 17:02:26.896080 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.895838    2570 scope.go:117] "RemoveContainer" containerID="9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c"
Jul 26 17:02:26.905354 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.905335    2570 scope.go:117] "RemoveContainer" containerID="9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c"
Jul 26 17:02:26.905594 ip-10-0-135-132 kubenswrapper[2570]: E0726 17:02:26.905576    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c\": container with ID starting with 9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c not found: ID does not exist" containerID="9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c"
Jul 26 17:02:26.905642 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.905602    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c"} err="failed to get container status \"9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c\": rpc error: code = NotFound desc = could not find container \"9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c\": container with ID starting with 9cf4c0bb6b030f561719cd26aa5adde43f007aed53ee78aaf4e780fa3e9ad83c not found: ID does not exist"
Jul 26 17:02:26.918209 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.918173    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"]
Jul 26 17:02:26.922248 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:26.922229    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-50e630/ipp-route-50e630-kserve-775979c467-hqmgh"]
Jul 26 17:02:28.300407 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:02:28.300378    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac865c22-7af2-4f1a-8d1c-ece22d3cd303" path="/var/lib/kubelet/pods/ac865c22-7af2-4f1a-8d1c-ece22d3cd303/volumes"
Jul 26 17:03:10.199591 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.199557    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"]
Jul 26 17:03:10.200079 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.199915    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66bb11d8-b453-49ad-8308-f47e1f2c9ccc" containerName="revoke-keys"
Jul 26 17:03:10.200079 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.199928    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="66bb11d8-b453-49ad-8308-f47e1f2c9ccc" containerName="revoke-keys"
Jul 26 17:03:10.200079 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.199945    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac865c22-7af2-4f1a-8d1c-ece22d3cd303" containerName="main"
Jul 26 17:03:10.200079 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.199951    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac865c22-7af2-4f1a-8d1c-ece22d3cd303" containerName="main"
Jul 26 17:03:10.200079 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.200012    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="66bb11d8-b453-49ad-8308-f47e1f2c9ccc" containerName="revoke-keys"
Jul 26 17:03:10.200079 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.200023    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac865c22-7af2-4f1a-8d1c-ece22d3cd303" containerName="main"
Jul 26 17:03:10.202896 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.202879    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"
Jul 26 17:03:10.205544 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.205520    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2xlcn\""
Jul 26 17:03:10.211231 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.211043    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"]
Jul 26 17:03:10.372692 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.372648    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw\" (UID: \"8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"
Jul 26 17:03:10.473802 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.473704    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw\" (UID: \"8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"
Jul 26 17:03:10.474373 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.474356    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw\" (UID: \"8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"
Jul 26 17:03:10.516372 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.516338    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"
Jul 26 17:03:10.648402 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:10.648374    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"]
Jul 26 17:03:10.651689 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:03:10.651660    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f15e3c4_0d0f_4045_a7e2_4bf304ea6e34.slice/crio-35a521ed1477f105e1ded942a95e767e31037c2cf578267f7ea721de1b8ada71 WatchSource:0}: Error finding container 35a521ed1477f105e1ded942a95e767e31037c2cf578267f7ea721de1b8ada71: Status 404 returned error can't find the container with id 35a521ed1477f105e1ded942a95e767e31037c2cf578267f7ea721de1b8ada71
Jul 26 17:03:11.107363 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:11.107327    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw" event={"ID":"8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34","Type":"ContainerStarted","Data":"4e75d97e8d75597a2b1624b6da7bd023e18ec0d15e1039c1dd6b99583b3c3b21"}
Jul 26 17:03:11.107535 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:11.107383    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw" event={"ID":"8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34","Type":"ContainerStarted","Data":"35a521ed1477f105e1ded942a95e767e31037c2cf578267f7ea721de1b8ada71"}
Jul 26 17:03:11.123363 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:11.123298    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw" podStartSLOduration=1.123281675 podStartE2EDuration="1.123281675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:03:11.122718866 +0000 UTC m=+3287.305803168" watchObservedRunningTime="2026-07-26 17:03:11.123281675 +0000 UTC m=+3287.306365961"
Jul 26 17:03:11.410727 ip-10-0-135-132 kubenswrapper[2570]: E0726 17:03:11.410695    2570 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f15e3c4_0d0f_4045_a7e2_4bf304ea6e34.slice/crio-conmon-4e75d97e8d75597a2b1624b6da7bd023e18ec0d15e1039c1dd6b99583b3c3b21.scope\": RecentStats: unable to find data in memory cache]"
Jul 26 17:03:12.113658 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:12.113627    2570 generic.go:356] "Generic (PLEG): container finished" podID="8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34" containerID="4e75d97e8d75597a2b1624b6da7bd023e18ec0d15e1039c1dd6b99583b3c3b21" exitCode=0
Jul 26 17:03:12.113835 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:12.113692    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw" event={"ID":"8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34","Type":"ContainerDied","Data":"4e75d97e8d75597a2b1624b6da7bd023e18ec0d15e1039c1dd6b99583b3c3b21"}
Jul 26 17:03:13.251742 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:13.251710    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"
Jul 26 17:03:13.401255 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:13.401134    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34-maas-api-service-ca\") pod \"8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34\" (UID: \"8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34\") "
Jul 26 17:03:13.401555 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:13.401527    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34" (UID: "8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:03:13.502453 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:13.502415    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34-maas-api-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:03:14.124258 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:14.124226    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"
Jul 26 17:03:14.124439 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:14.124250    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw" event={"ID":"8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34","Type":"ContainerDied","Data":"35a521ed1477f105e1ded942a95e767e31037c2cf578267f7ea721de1b8ada71"}
Jul 26 17:03:14.124439 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:14.124308    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35a521ed1477f105e1ded942a95e767e31037c2cf578267f7ea721de1b8ada71"
Jul 26 17:03:21.377478 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.377442    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"]
Jul 26 17:03:21.377955 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.377646    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp" podUID="f742be70-47f3-4523-8f52-aacc075b10fa" containerName="maas-api" containerID="cri-o://8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142" gracePeriod=30
Jul 26 17:03:21.411698 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.411667    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29"]
Jul 26 17:03:21.413654 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.413632    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-930b99-29751420-f7m29"]
Jul 26 17:03:21.583339 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.583253    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"]
Jul 26 17:03:21.583525 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.583485    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr" podUID="2747ef12-136d-4b51-8c1f-1d876b0d7be4" containerName="payload-processing" containerID="cri-o://68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2" gracePeriod=30
Jul 26 17:03:21.709338 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.709313    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 17:03:21.764397 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.764372    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnjwv\" (UniqueName: \"kubernetes.io/projected/f742be70-47f3-4523-8f52-aacc075b10fa-kube-api-access-fnjwv\") pod \"f742be70-47f3-4523-8f52-aacc075b10fa\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") "
Jul 26 17:03:21.764595 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.764416    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f742be70-47f3-4523-8f52-aacc075b10fa-openshift-service-ca\") pod \"f742be70-47f3-4523-8f52-aacc075b10fa\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") "
Jul 26 17:03:21.764595 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.764465    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f742be70-47f3-4523-8f52-aacc075b10fa-maas-api-tls\") pod \"f742be70-47f3-4523-8f52-aacc075b10fa\" (UID: \"f742be70-47f3-4523-8f52-aacc075b10fa\") "
Jul 26 17:03:21.764865 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.764839    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f742be70-47f3-4523-8f52-aacc075b10fa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f742be70-47f3-4523-8f52-aacc075b10fa" (UID: "f742be70-47f3-4523-8f52-aacc075b10fa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:03:21.768010 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.767987    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f742be70-47f3-4523-8f52-aacc075b10fa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f742be70-47f3-4523-8f52-aacc075b10fa" (UID: "f742be70-47f3-4523-8f52-aacc075b10fa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:03:21.768104 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.768083    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f742be70-47f3-4523-8f52-aacc075b10fa-kube-api-access-fnjwv" (OuterVolumeSpecName: "kube-api-access-fnjwv") pod "f742be70-47f3-4523-8f52-aacc075b10fa" (UID: "f742be70-47f3-4523-8f52-aacc075b10fa"). InnerVolumeSpecName "kube-api-access-fnjwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:03:21.840347 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.840323    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 17:03:21.865761 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.865734    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2747ef12-136d-4b51-8c1f-1d876b0d7be4-plugins-config-volume\") pod \"2747ef12-136d-4b51-8c1f-1d876b0d7be4\" (UID: \"2747ef12-136d-4b51-8c1f-1d876b0d7be4\") "
Jul 26 17:03:21.865925 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.865806    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsnbv\" (UniqueName: \"kubernetes.io/projected/2747ef12-136d-4b51-8c1f-1d876b0d7be4-kube-api-access-rsnbv\") pod \"2747ef12-136d-4b51-8c1f-1d876b0d7be4\" (UID: \"2747ef12-136d-4b51-8c1f-1d876b0d7be4\") "
Jul 26 17:03:21.866109 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.866092    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f742be70-47f3-4523-8f52-aacc075b10fa-maas-api-tls\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:03:21.866214 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.866112    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnjwv\" (UniqueName: \"kubernetes.io/projected/f742be70-47f3-4523-8f52-aacc075b10fa-kube-api-access-fnjwv\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:03:21.866214 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.866125    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f742be70-47f3-4523-8f52-aacc075b10fa-openshift-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:03:21.866214 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.866165    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2747ef12-136d-4b51-8c1f-1d876b0d7be4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2747ef12-136d-4b51-8c1f-1d876b0d7be4" (UID: "2747ef12-136d-4b51-8c1f-1d876b0d7be4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:03:21.867966 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.867940    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2747ef12-136d-4b51-8c1f-1d876b0d7be4-kube-api-access-rsnbv" (OuterVolumeSpecName: "kube-api-access-rsnbv") pod "2747ef12-136d-4b51-8c1f-1d876b0d7be4" (UID: "2747ef12-136d-4b51-8c1f-1d876b0d7be4"). InnerVolumeSpecName "kube-api-access-rsnbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:03:21.967304 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.967275    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsnbv\" (UniqueName: \"kubernetes.io/projected/2747ef12-136d-4b51-8c1f-1d876b0d7be4-kube-api-access-rsnbv\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:03:21.967304 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:21.967302    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2747ef12-136d-4b51-8c1f-1d876b0d7be4-plugins-config-volume\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:03:22.164729 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.164693    2570 generic.go:356] "Generic (PLEG): container finished" podID="f742be70-47f3-4523-8f52-aacc075b10fa" containerID="8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142" exitCode=0
Jul 26 17:03:22.164919 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.164748    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"
Jul 26 17:03:22.164919 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.164776    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp" event={"ID":"f742be70-47f3-4523-8f52-aacc075b10fa","Type":"ContainerDied","Data":"8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142"}
Jul 26 17:03:22.164919 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.164823    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp" event={"ID":"f742be70-47f3-4523-8f52-aacc075b10fa","Type":"ContainerDied","Data":"35fcffa8477d384cb161771cd109deacec3c5df225a5dde2d3ccb823bacd4cf0"}
Jul 26 17:03:22.164919 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.164849    2570 scope.go:117] "RemoveContainer" containerID="8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142"
Jul 26 17:03:22.166092 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.166073    2570 generic.go:356] "Generic (PLEG): container finished" podID="2747ef12-136d-4b51-8c1f-1d876b0d7be4" containerID="68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2" exitCode=0
Jul 26 17:03:22.166169 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.166115    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"
Jul 26 17:03:22.166169 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.166139    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr" event={"ID":"2747ef12-136d-4b51-8c1f-1d876b0d7be4","Type":"ContainerDied","Data":"68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2"}
Jul 26 17:03:22.166318 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.166198    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr" event={"ID":"2747ef12-136d-4b51-8c1f-1d876b0d7be4","Type":"ContainerDied","Data":"95c24883cde915e85f5cc6e316126a95325d97f64abe4a6b86b25f5fcb33e8ce"}
Jul 26 17:03:22.174400 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.174380    2570 scope.go:117] "RemoveContainer" containerID="8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142"
Jul 26 17:03:22.174686 ip-10-0-135-132 kubenswrapper[2570]: E0726 17:03:22.174665    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142\": container with ID starting with 8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142 not found: ID does not exist" containerID="8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142"
Jul 26 17:03:22.174756 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.174694    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142"} err="failed to get container status \"8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142\": rpc error: code = NotFound desc = could not find container \"8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142\": container with ID starting with 8dcc47786ca3bf62eaf297858c0370403860ef2086b0f30d22da0ff59ee6a142 not found: ID does not exist"
Jul 26 17:03:22.174756 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.174718    2570 scope.go:117] "RemoveContainer" containerID="68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2"
Jul 26 17:03:22.182391 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.182375    2570 scope.go:117] "RemoveContainer" containerID="68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2"
Jul 26 17:03:22.182625 ip-10-0-135-132 kubenswrapper[2570]: E0726 17:03:22.182607    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2\": container with ID starting with 68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2 not found: ID does not exist" containerID="68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2"
Jul 26 17:03:22.182675 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.182631    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2"} err="failed to get container status \"68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2\": rpc error: code = NotFound desc = could not find container \"68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2\": container with ID starting with 68b17884fc8d0b3db6936d6c0bf1a7e494cdb091bc3c2718325830a4eaeaf2d2 not found: ID does not exist"
Jul 26 17:03:22.187162 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.187138    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"]
Jul 26 17:03:22.189046 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.189027    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930b99-58756d98d8-f8wxp"]
Jul 26 17:03:22.200172 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.200151    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"]
Jul 26 17:03:22.207319 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.207300    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-930b99-5d4f494c84-j5lnr"]
Jul 26 17:03:22.300828 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.300755    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2747ef12-136d-4b51-8c1f-1d876b0d7be4" path="/var/lib/kubelet/pods/2747ef12-136d-4b51-8c1f-1d876b0d7be4/volumes"
Jul 26 17:03:22.301035 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.301022    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="677c510a-71d1-4d79-8f5d-0ec0b5c82184" path="/var/lib/kubelet/pods/677c510a-71d1-4d79-8f5d-0ec0b5c82184/volumes"
Jul 26 17:03:22.301348 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:22.301332    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f742be70-47f3-4523-8f52-aacc075b10fa" path="/var/lib/kubelet/pods/f742be70-47f3-4523-8f52-aacc075b10fa/volumes"
Jul 26 17:03:24.470543 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:24.470514    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 17:03:24.474472 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:24.474452    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn7k8_6d79cbbc-e4cb-4de2-ba7b-f2043363b9f5/ovn-acl-logging/0.log"
Jul 26 17:03:24.618105 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:24.618084    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 17:03:25.267634 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:25.267595    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"]
Jul 26 17:03:25.270055 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:25.270029    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-930b99-gr4sw"]
Jul 26 17:03:26.299671 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:26.299636    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34" path="/var/lib/kubelet/pods/8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34/volumes"
Jul 26 17:03:32.418856 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.418820    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"]
Jul 26 17:03:32.419288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.419165    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f742be70-47f3-4523-8f52-aacc075b10fa" containerName="maas-api"
Jul 26 17:03:32.419288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.419177    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f742be70-47f3-4523-8f52-aacc075b10fa" containerName="maas-api"
Jul 26 17:03:32.419288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.419205    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2747ef12-136d-4b51-8c1f-1d876b0d7be4" containerName="payload-processing"
Jul 26 17:03:32.419288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.419211    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2747ef12-136d-4b51-8c1f-1d876b0d7be4" containerName="payload-processing"
Jul 26 17:03:32.419288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.419218    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34" containerName="revoke-keys"
Jul 26 17:03:32.419288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.419224    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34" containerName="revoke-keys"
Jul 26 17:03:32.419288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.419280    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2747ef12-136d-4b51-8c1f-1d876b0d7be4" containerName="payload-processing"
Jul 26 17:03:32.419288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.419289    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f15e3c4-0d0f-4045-a7e2-4bf304ea6e34" containerName="revoke-keys"
Jul 26 17:03:32.419572 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.419295    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f742be70-47f3-4523-8f52-aacc075b10fa" containerName="maas-api"
Jul 26 17:03:32.423755 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.423739    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"
Jul 26 17:03:32.426728 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.426713    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 17:03:32.426812 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.426757    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2xlcn\""
Jul 26 17:03:32.433143 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.433120    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"]
Jul 26 17:03:32.467802 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.467770    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c036f13c-4662-41c4-b19f-f23b862283d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm\" (UID: \"c036f13c-4662-41c4-b19f-f23b862283d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"
Jul 26 17:03:32.569177 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.569140    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c036f13c-4662-41c4-b19f-f23b862283d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm\" (UID: \"c036f13c-4662-41c4-b19f-f23b862283d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"
Jul 26 17:03:32.569758 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.569740    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c036f13c-4662-41c4-b19f-f23b862283d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm\" (UID: \"c036f13c-4662-41c4-b19f-f23b862283d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"
Jul 26 17:03:32.734232 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.734181    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"
Jul 26 17:03:32.883593 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:32.883561    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"]
Jul 26 17:03:32.886576 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:03:32.886550    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc036f13c_4662_41c4_b19f_f23b862283d2.slice/crio-029426b0d2cbc764caa53cb099a70b2b2f5eeaae09b05750c2281cfc293d1b61 WatchSource:0}: Error finding container 029426b0d2cbc764caa53cb099a70b2b2f5eeaae09b05750c2281cfc293d1b61: Status 404 returned error can't find the container with id 029426b0d2cbc764caa53cb099a70b2b2f5eeaae09b05750c2281cfc293d1b61
Jul 26 17:03:33.221686 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:33.221647    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm" event={"ID":"c036f13c-4662-41c4-b19f-f23b862283d2","Type":"ContainerStarted","Data":"4d85794544d330b21e6f112888c9afcff73a3c6cafffd46c9f32f0b92593f80b"}
Jul 26 17:03:33.221881 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:33.221700    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm" event={"ID":"c036f13c-4662-41c4-b19f-f23b862283d2","Type":"ContainerStarted","Data":"029426b0d2cbc764caa53cb099a70b2b2f5eeaae09b05750c2281cfc293d1b61"}
Jul 26 17:03:33.237860 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:33.237811    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm" podStartSLOduration=1.23779814 podStartE2EDuration="1.23779814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:03:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:03:33.237372839 +0000 UTC m=+3309.420457148" watchObservedRunningTime="2026-07-26 17:03:33.23779814 +0000 UTC m=+3309.420882423"
Jul 26 17:03:34.228747 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:34.228713    2570 generic.go:356] "Generic (PLEG): container finished" podID="c036f13c-4662-41c4-b19f-f23b862283d2" containerID="4d85794544d330b21e6f112888c9afcff73a3c6cafffd46c9f32f0b92593f80b" exitCode=0
Jul 26 17:03:34.229131 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:34.228801    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm" event={"ID":"c036f13c-4662-41c4-b19f-f23b862283d2","Type":"ContainerDied","Data":"4d85794544d330b21e6f112888c9afcff73a3c6cafffd46c9f32f0b92593f80b"}
Jul 26 17:03:35.371529 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:35.371501    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"
Jul 26 17:03:35.493773 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:35.493690    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c036f13c-4662-41c4-b19f-f23b862283d2-maas-api-service-ca\") pod \"c036f13c-4662-41c4-b19f-f23b862283d2\" (UID: \"c036f13c-4662-41c4-b19f-f23b862283d2\") "
Jul 26 17:03:35.494078 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:35.494053    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c036f13c-4662-41c4-b19f-f23b862283d2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c036f13c-4662-41c4-b19f-f23b862283d2" (UID: "c036f13c-4662-41c4-b19f-f23b862283d2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:03:35.594916 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:35.594872    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c036f13c-4662-41c4-b19f-f23b862283d2-maas-api-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:03:36.245077 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:36.245044    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"
Jul 26 17:03:36.245277 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:36.245044    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm" event={"ID":"c036f13c-4662-41c4-b19f-f23b862283d2","Type":"ContainerDied","Data":"029426b0d2cbc764caa53cb099a70b2b2f5eeaae09b05750c2281cfc293d1b61"}
Jul 26 17:03:36.245277 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:36.245169    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="029426b0d2cbc764caa53cb099a70b2b2f5eeaae09b05750c2281cfc293d1b61"
Jul 26 17:03:43.956382 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:43.956348    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm"]
Jul 26 17:03:43.959067 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:43.959043    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1b2072-29751420-zwtsm"]
Jul 26 17:03:44.300769 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:44.300682    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4e81ed2-1773-4760-989a-ff81685d0426" path="/var/lib/kubelet/pods/a4e81ed2-1773-4760-989a-ff81685d0426/volumes"
Jul 26 17:03:46.920531 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:46.920455    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 17:03:47.492959 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:47.492923    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"]
Jul 26 17:03:47.494448 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:47.494423    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b2072-mndwm"]
Jul 26 17:03:48.299688 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:48.299646    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c036f13c-4662-41c4-b19f-f23b862283d2" path="/var/lib/kubelet/pods/c036f13c-4662-41c4-b19f-f23b862283d2/volumes"
Jul 26 17:03:54.861208 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:54.861164    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"]
Jul 26 17:03:54.861600 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:54.861487    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c036f13c-4662-41c4-b19f-f23b862283d2" containerName="revoke-keys"
Jul 26 17:03:54.861600 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:54.861497    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c036f13c-4662-41c4-b19f-f23b862283d2" containerName="revoke-keys"
Jul 26 17:03:54.861600 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:54.861565    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c036f13c-4662-41c4-b19f-f23b862283d2" containerName="revoke-keys"
Jul 26 17:03:54.865560 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:54.865543    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"
Jul 26 17:03:54.868467 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:54.868440    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2xlcn\""
Jul 26 17:03:54.868585 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:54.868449    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 17:03:54.873080 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:54.873058    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"]
Jul 26 17:03:54.972841 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:54.972811    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ea231aa-7f7b-423b-aad2-7cc371cb81a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a06722-6tr74\" (UID: \"2ea231aa-7f7b-423b-aad2-7cc371cb81a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"
Jul 26 17:03:55.073866 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:55.073827    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ea231aa-7f7b-423b-aad2-7cc371cb81a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a06722-6tr74\" (UID: \"2ea231aa-7f7b-423b-aad2-7cc371cb81a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"
Jul 26 17:03:55.074442 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:55.074422    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ea231aa-7f7b-423b-aad2-7cc371cb81a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a06722-6tr74\" (UID: \"2ea231aa-7f7b-423b-aad2-7cc371cb81a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"
Jul 26 17:03:55.176521 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:55.176443    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"
Jul 26 17:03:55.305592 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:55.305567    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"]
Jul 26 17:03:55.308167 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:03:55.308138    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ea231aa_7f7b_423b_aad2_7cc371cb81a3.slice/crio-c38b775f6ae5729f1c3348d50b738933e0e6ad9d49d0774e57f66fcb87ab9abd WatchSource:0}: Error finding container c38b775f6ae5729f1c3348d50b738933e0e6ad9d49d0774e57f66fcb87ab9abd: Status 404 returned error can't find the container with id c38b775f6ae5729f1c3348d50b738933e0e6ad9d49d0774e57f66fcb87ab9abd
Jul 26 17:03:55.335227 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:55.335202    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74" event={"ID":"2ea231aa-7f7b-423b-aad2-7cc371cb81a3","Type":"ContainerStarted","Data":"c38b775f6ae5729f1c3348d50b738933e0e6ad9d49d0774e57f66fcb87ab9abd"}
Jul 26 17:03:56.341142 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:56.341055    2570 generic.go:356] "Generic (PLEG): container finished" podID="2ea231aa-7f7b-423b-aad2-7cc371cb81a3" containerID="7e6df1e5a50e9ee88818c6d0e6c5617498e0ea310aae7ecd0f76a5d661de866d" exitCode=0
Jul 26 17:03:56.341142 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:56.341120    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74" event={"ID":"2ea231aa-7f7b-423b-aad2-7cc371cb81a3","Type":"ContainerDied","Data":"7e6df1e5a50e9ee88818c6d0e6c5617498e0ea310aae7ecd0f76a5d661de866d"}
Jul 26 17:03:57.474459 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:57.474435    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"
Jul 26 17:03:57.595851 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:57.595824    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ea231aa-7f7b-423b-aad2-7cc371cb81a3-maas-api-service-ca\") pod \"2ea231aa-7f7b-423b-aad2-7cc371cb81a3\" (UID: \"2ea231aa-7f7b-423b-aad2-7cc371cb81a3\") "
Jul 26 17:03:57.596256 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:57.596177    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ea231aa-7f7b-423b-aad2-7cc371cb81a3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ea231aa-7f7b-423b-aad2-7cc371cb81a3" (UID: "2ea231aa-7f7b-423b-aad2-7cc371cb81a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:03:57.697004 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:57.696970    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ea231aa-7f7b-423b-aad2-7cc371cb81a3-maas-api-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\""
Jul 26 17:03:58.355627 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:58.355599    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"
Jul 26 17:03:58.355786 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:58.355594    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74" event={"ID":"2ea231aa-7f7b-423b-aad2-7cc371cb81a3","Type":"ContainerDied","Data":"c38b775f6ae5729f1c3348d50b738933e0e6ad9d49d0774e57f66fcb87ab9abd"}
Jul 26 17:03:58.355786 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:03:58.355751    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c38b775f6ae5729f1c3348d50b738933e0e6ad9d49d0774e57f66fcb87ab9abd"
Jul 26 17:04:05.653615 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:05.653575    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d"]
Jul 26 17:04:05.657751 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:05.657722    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a06722-29751420-2qf7d"]
Jul 26 17:04:06.300167 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:06.300131    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b7f7f4-6a2f-4c54-8a7b-fd8ed76610f7" path="/var/lib/kubelet/pods/e6b7f7f4-6a2f-4c54-8a7b-fd8ed76610f7/volumes"
Jul 26 17:04:08.514964 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:08.514929    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6nbh"]
Jul 26 17:04:09.929858 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:09.929800    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"]
Jul 26 17:04:09.931906 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:09.931886    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a06722-6tr74"]
Jul 26 17:04:10.300265 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:10.300228    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ea231aa-7f7b-423b-aad2-7cc371cb81a3" path="/var/lib/kubelet/pods/2ea231aa-7f7b-423b-aad2-7cc371cb81a3/volumes"
Jul 26 17:04:17.032059 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:17.032023    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d795fc685-jbzp6_a4521ccb-7dc2-43e9-98d4-3e646f571cf1/authorino/0.log"
Jul 26 17:04:22.339254 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:22.339225    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xt7ng_d9486caa-a9f7-4f10-a6af-2a64e508313c/manager/0.log"
Jul 26 17:04:23.071687 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:23.071656    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd4986cdb-kz98g_1ff7b7b9-0c36-4233-aadb-1fe983e267e2/copy-manifests/0.log"
Jul 26 17:04:23.078773 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:23.078750    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd4986cdb-kz98g_1ff7b7b9-0c36-4233-aadb-1fe983e267e2/manager/0.log"
Jul 26 17:04:23.632462 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:23.632433    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751405-jnr7l_65d3a462-332d-43a2-9928-79dfc049841a/cleanup/0.log"
Jul 26 17:04:23.746643 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:23.746619    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751420-hgvnf_f285656e-c485-4685-8c9d-62c097c187fe/cleanup/0.log"
Jul 26 17:04:25.112387 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:25.112343    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d795fc685-jbzp6_a4521ccb-7dc2-43e9-98d4-3e646f571cf1/authorino/0.log"
Jul 26 17:04:25.781553 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:25.781527    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b6nbh_22668da6-cd04-4a5c-9c24-1a33de4b9f55/limitador/0.log"
Jul 26 17:04:26.465231 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:26.465183    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6ff4cb76d8-lbtrd_1c823567-6693-49d6-bed4-96472ee97dec/kube-auth-proxy/0.log"
Jul 26 17:04:27.483082 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:27.483048    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-f6scz_49b021e8-5109-443c-9304-e9bfc4ba9365/main/0.log"
Jul 26 17:04:27.829292 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:27.829223    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-9xc64_8657640b-1cdb-40fc-b83d-5898af353b1c/main/0.log"
Jul 26 17:04:27.940285 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:27.940254    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-q86lp_97b0301b-3fe5-494a-8820-9ee5fbc18277/main/0.log"
Jul 26 17:04:36.165552 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:36.165521    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5dfgr_15961897-ec82-41cf-8ce5-b036a30b987c/global-pull-secret-syncer/0.log"
Jul 26 17:04:36.382666 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:36.382632    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lxsxf_74387622-269d-4e67-af29-776fabd1e176/konnectivity-agent/0.log"
Jul 26 17:04:36.445798 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:36.445771    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-132.ec2.internal_a3de1b13181d39474231213cb35e8b2d/haproxy/0.log"
Jul 26 17:04:41.244889 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:41.244858    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d795fc685-jbzp6_a4521ccb-7dc2-43e9-98d4-3e646f571cf1/authorino/0.log"
Jul 26 17:04:41.657292 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:41.657216    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b6nbh_22668da6-cd04-4a5c-9c24-1a33de4b9f55/limitador/0.log"
Jul 26 17:04:43.802558 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:43.802531    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-76b54c898d-8nvw2_2c6ec0cf-f3ee-484a-94f7-81a0298e2beb/metrics-server/0.log"
Jul 26 17:04:44.067824 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.067749    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t4t5l_5328de77-0706-4e80-b992-51512fa49017/node-exporter/0.log"
Jul 26 17:04:44.090904 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.090880    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t4t5l_5328de77-0706-4e80-b992-51512fa49017/kube-rbac-proxy/0.log"
Jul 26 17:04:44.113424 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.113337    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t4t5l_5328de77-0706-4e80-b992-51512fa49017/init-textfile/0.log"
Jul 26 17:04:44.247936 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.247898    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ca56160f-ab52-45c8-b99f-1f5f60723672/prometheus/0.log"
Jul 26 17:04:44.269904 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.269883    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ca56160f-ab52-45c8-b99f-1f5f60723672/config-reloader/0.log"
Jul 26 17:04:44.295526 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.295497    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ca56160f-ab52-45c8-b99f-1f5f60723672/thanos-sidecar/0.log"
Jul 26 17:04:44.318481 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.318410    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ca56160f-ab52-45c8-b99f-1f5f60723672/kube-rbac-proxy-web/0.log"
Jul 26 17:04:44.339776 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.339752    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ca56160f-ab52-45c8-b99f-1f5f60723672/kube-rbac-proxy/0.log"
Jul 26 17:04:44.361478 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.361457    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ca56160f-ab52-45c8-b99f-1f5f60723672/kube-rbac-proxy-thanos/0.log"
Jul 26 17:04:44.382658 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.382637    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ca56160f-ab52-45c8-b99f-1f5f60723672/init-config-reloader/0.log"
Jul 26 17:04:44.410288 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.410263    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-2zqnt_532db9ce-7850-4661-90e7-a6fd3d29dad5/prometheus-operator/0.log"
Jul 26 17:04:44.429461 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.429439    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-2zqnt_532db9ce-7850-4661-90e7-a6fd3d29dad5/kube-rbac-proxy/0.log"
Jul 26 17:04:44.458649 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.458631    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-gcptj_6e170623-cf7e-4197-b340-47661c2a1597/prometheus-operator-admission-webhook/0.log"
Jul 26 17:04:44.561054 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.561028    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66dbbcf55f-b8hqx_d2b70c7c-86b7-430d-8a6c-07122090e03d/thanos-query/0.log"
Jul 26 17:04:44.582980 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.582913    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66dbbcf55f-b8hqx_d2b70c7c-86b7-430d-8a6c-07122090e03d/kube-rbac-proxy-web/0.log"
Jul 26 17:04:44.604110 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.604087    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66dbbcf55f-b8hqx_d2b70c7c-86b7-430d-8a6c-07122090e03d/kube-rbac-proxy/0.log"
Jul 26 17:04:44.626899 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.626876    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66dbbcf55f-b8hqx_d2b70c7c-86b7-430d-8a6c-07122090e03d/prom-label-proxy/0.log"
Jul 26 17:04:44.649971 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.649953    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66dbbcf55f-b8hqx_d2b70c7c-86b7-430d-8a6c-07122090e03d/kube-rbac-proxy-rules/0.log"
Jul 26 17:04:44.674172 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:44.674154    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66dbbcf55f-b8hqx_d2b70c7c-86b7-430d-8a6c-07122090e03d/kube-rbac-proxy-metrics/0.log"
Jul 26 17:04:46.069933 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.069900    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"]
Jul 26 17:04:46.070304 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.070258    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ea231aa-7f7b-423b-aad2-7cc371cb81a3" containerName="revoke-keys"
Jul 26 17:04:46.070304 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.070270    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ea231aa-7f7b-423b-aad2-7cc371cb81a3" containerName="revoke-keys"
Jul 26 17:04:46.070377 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.070320    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ea231aa-7f7b-423b-aad2-7cc371cb81a3" containerName="revoke-keys"
Jul 26 17:04:46.073337 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.073321    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.076456 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.076426    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bnlmg\"/\"openshift-service-ca.crt\""
Jul 26 17:04:46.076456 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.076452    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bnlmg\"/\"default-dockercfg-g2bhf\""
Jul 26 17:04:46.077659 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.077644    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bnlmg\"/\"kube-root-ca.crt\""
Jul 26 17:04:46.090129 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.090100    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"]
Jul 26 17:04:46.212939 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.212896    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-lib-modules\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.213150 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.212955    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-podres\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.213150 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.213059    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27ldv\" (UniqueName: \"kubernetes.io/projected/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-kube-api-access-27ldv\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.213329 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.213152    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-sys\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.213329 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.213215    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-proc\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.314135 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.314103    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-sys\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.314135 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.314138    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-proc\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.314381 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.314175    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-lib-modules\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.314381 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.314209    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-podres\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.314381 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.314245    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-proc\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.314381 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.314239    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27ldv\" (UniqueName: \"kubernetes.io/projected/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-kube-api-access-27ldv\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.314381 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.314246    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-sys\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.314381 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.314312    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-podres\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.314381 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.314320    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-lib-modules\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.323557 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.323504    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27ldv\" (UniqueName: \"kubernetes.io/projected/c5a7eecd-53c7-44ee-ac0c-81da71cec3f4-kube-api-access-27ldv\") pod \"perf-node-gather-daemonset-5cdjg\" (UID: \"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4\") " pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.382939 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.382910    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:46.510560 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.510533    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"]
Jul 26 17:04:46.512691 ip-10-0-135-132 kubenswrapper[2570]: W0726 17:04:46.512658    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc5a7eecd_53c7_44ee_ac0c_81da71cec3f4.slice/crio-93c79ba33e043207d731e7c8fa3d84820b03a42223fa5cbce08562ba9e73ca04 WatchSource:0}: Error finding container 93c79ba33e043207d731e7c8fa3d84820b03a42223fa5cbce08562ba9e73ca04: Status 404 returned error can't find the container with id 93c79ba33e043207d731e7c8fa3d84820b03a42223fa5cbce08562ba9e73ca04
Jul 26 17:04:46.585724 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:46.585694    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg" event={"ID":"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4","Type":"ContainerStarted","Data":"93c79ba33e043207d731e7c8fa3d84820b03a42223fa5cbce08562ba9e73ca04"}
Jul 26 17:04:47.592053 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:47.592023    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg" event={"ID":"c5a7eecd-53c7-44ee-ac0c-81da71cec3f4","Type":"ContainerStarted","Data":"0a2f3c0da98b72175ccd6a1a6ea5f3c97c0e792797c1f4b46fda6117dd0cf21b"}
Jul 26 17:04:47.592488 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:47.592085    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:47.607615 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:47.607565    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg" podStartSLOduration=1.607547294 podStartE2EDuration="1.607547294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:04:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:04:47.606713942 +0000 UTC m=+3383.789798251" watchObservedRunningTime="2026-07-26 17:04:47.607547294 +0000 UTC m=+3383.790631582"
Jul 26 17:04:48.926741 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:48.926715    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tnrlp_411592b1-9f45-453b-a163-f9fdd81b9ef3/dns/0.log"
Jul 26 17:04:48.948890 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:48.948868    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tnrlp_411592b1-9f45-453b-a163-f9fdd81b9ef3/kube-rbac-proxy/0.log"
Jul 26 17:04:49.026631 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:49.026607    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6f5mx_01f4e8e6-8355-42ec-9f7a-33c4f0d68c90/dns-node-resolver/0.log"
Jul 26 17:04:49.620447 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:49.620420    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b59467984-gbq79_4464b0fb-0684-4a0c-b3c4-3cdbd643ffc8/registry/0.log"
Jul 26 17:04:49.666866 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:49.666843    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nb5cd_e0b75180-bc1f-42fa-9792-c2de53a22d56/node-ca/0.log"
Jul 26 17:04:50.865914 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:50.865882    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6ff4cb76d8-lbtrd_1c823567-6693-49d6-bed4-96472ee97dec/kube-auth-proxy/0.log"
Jul 26 17:04:51.945217 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:51.945174    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t9gsq_85d1cceb-8043-4994-9bfe-8c690d05319e/serve-healthcheck-canary/0.log"
Jul 26 17:04:52.589622 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:52.589591    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g48ff_b089bff1-c417-441f-9921-521378315b57/kube-rbac-proxy/0.log"
Jul 26 17:04:52.613134 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:52.613106    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g48ff_b089bff1-c417-441f-9921-521378315b57/exporter/0.log"
Jul 26 17:04:52.635314 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:52.635290    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g48ff_b089bff1-c417-441f-9921-521378315b57/extractor/0.log"
Jul 26 17:04:53.607002 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:53.606977    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bnlmg/perf-node-gather-daemonset-5cdjg"
Jul 26 17:04:55.073341 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:55.073305    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xt7ng_d9486caa-a9f7-4f10-a6af-2a64e508313c/manager/0.log"
Jul 26 17:04:55.685484 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:55.685398    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd4986cdb-kz98g_1ff7b7b9-0c36-4233-aadb-1fe983e267e2/manager/0.log"
Jul 26 17:04:55.723923 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:55.723899    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd4986cdb-kz98g_1ff7b7b9-0c36-4233-aadb-1fe983e267e2/copy-manifests/0.log"
Jul 26 17:04:56.814350 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:56.814312    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6ff4cb76d8-lbtrd_1c823567-6693-49d6-bed4-96472ee97dec/kube-auth-proxy/0.log"
Jul 26 17:04:57.143382 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:57.143303    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76c849bdf9-jsslf_cfd9f872-0925-4839-8790-9453c11c1a63/manager/0.log"
Jul 26 17:04:57.172580 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:57.172556    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hlzx6_18e4cbe3-ab70-4a6e-85f3-3de5ab613a02/openshift-lws-operator/0.log"
Jul 26 17:04:58.552300 ip-10-0-135-132 kubenswrapper[2570]: I0726 17:04:58.552267    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6ff4cb76d8-lbtrd_1c823567-6693-49d6-bed4-96472ee97dec/kube-auth-proxy/0.log"