Jul 14 17:02:57.485016 ip-10-0-137-134 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 14 17:02:57.485027 ip-10-0-137-134 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 14 17:02:57.485034 ip-10-0-137-134 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 14 17:02:57.485326 ip-10-0-137-134 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 14 17:03:07.671079 ip-10-0-137-134 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 14 17:03:07.671099 ip-10-0-137-134 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot dd2f4bd1a02646b6bf996261d93f4864 --
Jul 14 17:05:13.122219 ip-10-0-137-134 systemd[1]: Starting Kubernetes Kubelet...
Jul 14 17:05:13.562462 ip-10-0-137-134 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 17:05:13.562462 ip-10-0-137-134 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 14 17:05:13.562462 ip-10-0-137-134 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 17:05:13.562462 ip-10-0-137-134 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 14 17:05:13.562462 ip-10-0-137-134 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 17:05:13.563605 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.563510    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 14 17:05:13.572270 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572240    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:05:13.572270 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572263    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:05:13.572270 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572267    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:05:13.572270 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572271    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:05:13.572270 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572273    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:05:13.572270 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572276    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:05:13.572270 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572280    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:05:13.572270 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572282    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572285    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572288    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572294    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572298    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572301    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572304    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572307    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572310    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572312    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572315    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572318    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572320    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572323    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572332    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572336    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572339    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572342    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572344    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572347    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:05:13.572555 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572349    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572352    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572354    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572357    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572359    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572362    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572364    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572367    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572370    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572372    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572375    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572377    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572380    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572383    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572386    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572388    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572391    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572394    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572397    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572400    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:05:13.573054 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572402    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572406    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572411    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572415    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572419    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572422    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572425    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572428    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572430    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572433    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572435    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572438    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572440    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572443    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572446    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572449    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572451    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572454    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:05:13.573540 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572456    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572459    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572463    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572465    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572468    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572470    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572473    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572476    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572480    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572483    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572485    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572488    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572490    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572493    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572496    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572498    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572501    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572504    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572512    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572515    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:05:13.574008 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572517    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572520    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572523    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572525    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572528    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572530    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572533    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572536    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572539    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572541    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572543    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572546    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572548    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.572551    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573274    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573280    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573283    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573286    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573289    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573291    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:05:13.574491 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573294    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573298    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573301    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573303    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573306    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573309    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573311    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573314    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573317    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573319    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573322    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573325    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573328    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573331    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573333    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573336    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573338    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573341    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573345    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:05:13.575017 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573349    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573352    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573355    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573357    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573360    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573363    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573366    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573368    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573371    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573374    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573377    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573380    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573384    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573388    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573391    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573395    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573398    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573402    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:05:13.575492 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573404    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573407    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573410    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573412    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573415    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573417    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573420    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573424    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573427    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573430    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573432    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573435    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573438    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573440    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573443    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573445    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573448    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573450    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573453    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573455    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:05:13.575961 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573458    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573461    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573464    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573467    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573469    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573472    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573474    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573477    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573479    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573482    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573484    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573487    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573489    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573492    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573495    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573497    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573500    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573502    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573505    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573507    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:05:13.576453 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573510    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573512    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573515    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573519    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573522    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573524    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573526    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573529    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573532    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573534    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573536    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573539    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573541    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573546    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573549    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573551    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573970    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573977    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573981    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:05:13.576991 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573985    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573988    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573991    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573994    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.573997    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574000    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574002    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574006    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574010    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574013    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574016    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574019    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574021    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574025    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574028    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574030    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574033    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574037    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574041    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:05:13.577445 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574044    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574047    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574050    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574053    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574056    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574058    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574061    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574065    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574067    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574070    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574072    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574075    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574077    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574080    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574082    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574085    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574087    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574089    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574092    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574094    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:05:13.577939 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574097    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574100    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574102    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574105    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574107    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574109    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574112    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574115    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574118    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574121    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574123    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574125    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574128    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574131    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574133    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574135    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574138    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574140    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574143    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574146    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:05:13.578419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574148    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574152    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574155    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574157    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574160    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574162    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574165    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574168    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574170    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574173    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574176    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574178    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574181    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574183    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574186    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574188    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574190    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574193    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574195    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:05:13.578921 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574198    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574203    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574205    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574208    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574210    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574213    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574215    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574218    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574220    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574222    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574225    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574228    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574230    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574233    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574235    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574240    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574242    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574245    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574315    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574327    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574335    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574342    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 14 17:05:13.579391 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574350    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574356    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574361    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574366    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574369    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574373    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574376    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574380    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574383    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574386    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574389    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574392    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574396    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574399    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574404    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574407    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574410    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574413    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574416    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574420    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574423    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574426    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574429    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574433    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 14 17:05:13.579977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574436    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574441    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574444    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574447    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574451    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574454    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574457    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574460    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574463    2580 flags.go:64] FLAG: --enable-server="true"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574466    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574470    2580 flags.go:64] FLAG: --event-burst="100"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574473    2580 flags.go:64] FLAG: --event-qps="50"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574476    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574479    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574482    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574486    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574489    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574492    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574495    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574498    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574501    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574505    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574508    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574511    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574514    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 14 17:05:13.580553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574516    2580 flags.go:64] FLAG: --feature-gates=""
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574520    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574525    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574528    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574531    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574534    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574537    2580 flags.go:64] FLAG: --help="false"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574540    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-137-134.ec2.internal"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574545    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574548    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574551    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574554    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574557    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574560    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574563    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574566    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574569    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574572    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574575    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574578    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574581    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574584    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574587    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574589    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 14 17:05:13.581201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574592    2580 flags.go:64] FLAG: --lock-file=""
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574595    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574598    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574600    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574606    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574609    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574612    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574615    2580 flags.go:64] FLAG: --logging-format="text"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574618    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574621    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574625    2580 flags.go:64] FLAG: --manifest-url=""
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574628    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574632    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574635    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574640    2580 flags.go:64] FLAG: --max-pods="110"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574643    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574648    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574651    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574654    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574657    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574660    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574662    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574670    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574674    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574677    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 14 17:05:13.581797 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574680    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574683    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574688    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574691    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574694    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574697    2580 flags.go:64] FLAG: --port="10250"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574700    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574702    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-039b1780ce57898cc"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574706    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574709    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574712    2580 flags.go:64] FLAG: --register-node="true"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574715    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574718    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574722    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574725    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574728    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574732    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574735    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574739    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574742    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574745    2580 flags.go:64] FLAG: --runonce="false"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574748    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574752    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574756    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574759    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 14 17:05:13.582432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574762    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574765    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574768    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574771    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574774    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574777    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574780    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574783    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574786    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574789    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574792    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574797    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574800    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574803    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574807    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574810    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574813    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574816    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574819    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574822    2580 flags.go:64] FLAG: --v="2"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574826    2580 flags.go:64] FLAG: --version="false"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574831    2580 flags.go:64] FLAG: --vmodule=""
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574835    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.574838    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574983    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:05:13.583084 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574988    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574991    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574993    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.574997    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575000    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575004    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575007    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575010    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575013    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575015    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575018    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575022    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575026    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575030    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575032    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575035    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575038    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575040    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575043    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:05:13.583734 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575047    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575050    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575054    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575056    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575059    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575063    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575065    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575068    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575071    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575074    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575077    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575079    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575082    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575086    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575089    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575091    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575094    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575096    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575100    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575103    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:05:13.584222 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575106    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575108    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575111    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575114    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575116    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575119    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575121    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575124    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575126    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575129    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575131    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575134    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575136    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575139    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575141    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575144    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575146    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575148    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575151    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575153    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575156    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:05:13.584717 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575159    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575162    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575173    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575176    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575180    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575183    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575186    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575189    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575192    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575196    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575198    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575201    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575204    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575207    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575209    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575212    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575214    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575217    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575219    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575222    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:05:13.585242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575225    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575227    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575229    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575232    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575235    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575237    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575240    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575242    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575244    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575247    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575249    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575252    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575260    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575263    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575265    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575273    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575277    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:05:13.585729 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.575279    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.575822    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.583275    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.583393    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.584466    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.584474    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584537    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584542    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584547    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584550    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584553    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584556    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584558    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584561    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584564    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:05:13.586186 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584567    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584569    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584572    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584575    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584578    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584580    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584583    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584585    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584589    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584593    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584596    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584598    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584601    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584603    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584606    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584608    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584611    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584614    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584617    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:05:13.586568 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584620    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584624    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584627    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584630    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584633    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584635    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584638    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584641    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584644    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584647    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584649    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584652    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584655    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584657    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584660    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584662    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584665    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584667    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584670    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:05:13.587089 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584674    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584678    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584681    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584684    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584686    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584689    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584692    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584694    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584697    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584699    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584702    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584704    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584707    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584709    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584713    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584717    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584719    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584722    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584725    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584728    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:05:13.587557 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584731    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584734    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584736    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584739    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584741    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584744    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584746    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584749    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584751    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584754    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584756    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584759    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584762    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584764    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584767    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584769    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584772    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584776    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584779    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:05:13.588080 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584782    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584785    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584788    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584790    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584793    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584795    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584798    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584800    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584803    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584806    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584809    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584812    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584814    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.584820    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584943    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:05:13.588537 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584947    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584950    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584953    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584956    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584958    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584961    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584964    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584967    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584969    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584972    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584975    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584978    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584980    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584983    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584985    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584988    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584991    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584993    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584996    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.584998    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:05:13.588930 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585001    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585004    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585007    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585009    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585012    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585015    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585018    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585021    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585023    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585026    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585028    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585031    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585033    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585035    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585038    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585040    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585043    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585045    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585048    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:05:13.589419 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585050    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585054    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585057    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585060    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585062    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585065    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585069    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585072    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585075    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585078    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585080    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585083    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585085    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585088    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585090    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585093    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585095    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585098    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585101    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:05:13.589923 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585103    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585106    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585109    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585111    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585114    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585117    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585119    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585122    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585124    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585127    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585129    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585132    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585134    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585137    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585140    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585142    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585145    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585147    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585150    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585152    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:05:13.590398 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585154    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585157    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585159    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585162    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585164    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585167    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585169    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585172    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585175    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585177    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585180    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585183    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585185    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585188    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585190    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585193    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585197    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585200    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585203    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:05:13.591053 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:13.585206    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:05:13.591519 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.585212    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 17:05:13.591519 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.585324    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 14 17:05:13.591519 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.587660    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 14 17:05:13.591519 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.589087    2580 server.go:1019] "Starting client certificate rotation"
Jul 14 17:05:13.591519 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.589192    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 17:05:13.591519 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.590000    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 17:05:13.592000 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.591984    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 17:05:13.596189 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.596164    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 17:05:13.608925 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.608902    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 14 17:05:13.611152 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.611135    2580 log.go:25] "Validated CRI v1 image API"
Jul 14 17:05:13.611922 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.611907    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 14 17:05:13.616430 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.616406    2580 fs.go:135] Filesystem UUIDs: map[3f7fd85e-b219-43ac-a818-a89799830b49:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e4517a35-d0d5-479c-a005-b1d52b52f492:/dev/nvme0n1p3]
Jul 14 17:05:13.616503 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.616429    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 14 17:05:13.617517 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.617501    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 17:05:13.622893 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.622736    2580 manager.go:216] Machine: {Timestamp:2026-07-14 17:05:13.621265051 +0000 UTC m=+0.368864939 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109848 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2174ce0d7c1be8bccbfb072b8a592e SystemUUID:ec2174ce-0d7c-1be8-bccb-fb072b8a592e BootID:dd2f4bd1-a026-46b6-bf99-6261d93f4864 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f3:79:1b:8e:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f3:79:1b:8e:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:1c:de:e7:a9:7b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 14 17:05:13.622893 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.622891    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 14 17:05:13.623001 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.622988    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 14 17:05:13.626675 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.626645    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 14 17:05:13.626816 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.626678    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-134.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 14 17:05:13.627438 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.627427    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 14 17:05:13.627472 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.627439    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 14 17:05:13.627472 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.627448    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 17:05:13.627472 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.627463    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 17:05:13.628837 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.628825    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 14 17:05:13.631588 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.631573    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 17:05:13.631718 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.631708    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 14 17:05:13.631756 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.631741    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 14 17:05:13.631756 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.631752    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 14 17:05:13.631808 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.631763    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 14 17:05:13.631808 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.631774    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 14 17:05:13.631808 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.631784    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 14 17:05:13.632242 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.632228    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 17:05:13.632296 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.632250    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 17:05:13.635802 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.635734    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 14 17:05:13.638377 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.638360    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 14 17:05:13.638447 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.638392    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 14 17:05:13.638580 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.638569    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 14 17:05:13.638619 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.638585    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 14 17:05:13.638619 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.638591    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 14 17:05:13.638619 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.638597    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 14 17:05:13.638619 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.638605    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 14 17:05:13.638619 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.638611    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 14 17:05:13.639243 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.639232    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 14 17:05:13.639243 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.639243    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 14 17:05:13.639324 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.639251    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 14 17:05:13.639324 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.639256    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 14 17:05:13.639324 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.639265    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 14 17:05:13.639324 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.639274    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 14 17:05:13.640502 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.640493    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 14 17:05:13.640502 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.640502    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 14 17:05:13.641453 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.641425    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 14 17:05:13.641523 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.641485    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 14 17:05:13.643994 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.643980    2580 server.go:1268] "Started kubelet"
Jul 14 17:05:13.644139 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.644113    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 14 17:05:13.644727 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.644092    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 14 17:05:13.644727 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.644322    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 14 17:05:13.645152 ip-10-0-137-134 systemd[1]: Started Kubernetes Kubelet.
Jul 14 17:05:13.645778 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.645596    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 14 17:05:13.646186 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.646172    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 14 17:05:13.650716 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.650694    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n7dpx"
Jul 14 17:05:13.651032 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.651014    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 14 17:05:13.651100 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.651014    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 14 17:05:13.651150 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.651137    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 14 17:05:13.651307 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.651281    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:13.651918 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.651900    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 14 17:05:13.651918 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.651915    2580 factory.go:55] Registering systemd factory
Jul 14 17:05:13.651918 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.651922    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 14 17:05:13.652267 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.651968    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 14 17:05:13.652267 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.652095    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 14 17:05:13.652267 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.652236    2580 factory.go:153] Registering CRI-O factory
Jul 14 17:05:13.652267 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.652251    2580 factory.go:223] Registration of the crio container factory successfully
Jul 14 17:05:13.652441 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.652296    2580 factory.go:103] Registering Raw factory
Jul 14 17:05:13.652441 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.652313    2580 manager.go:1196] Started watching for new ooms in manager
Jul 14 17:05:13.652681 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.652661    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-134.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 14 17:05:13.652983 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.652968    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 14 17:05:13.653113 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.653101    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 14 17:05:13.653113 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.653113    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 14 17:05:13.653227 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.653146    2580 manager.go:349] Starting recovery of all containers
Jul 14 17:05:13.653873 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.652772    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-134.ec2.internal.18c236da4ebd8a99  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-134.ec2.internal,UID:ip-10-0-137-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-134.ec2.internal,},FirstTimestamp:2026-07-14 17:05:13.643952793 +0000 UTC m=+0.391552664,LastTimestamp:2026-07-14 17:05:13.643952793 +0000 UTC m=+0.391552664,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-134.ec2.internal,}"
Jul 14 17:05:13.656373 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.656341    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 14 17:05:13.656496 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.656474    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 14 17:05:13.660115 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.660089    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n7dpx"
Jul 14 17:05:13.660556 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.660507    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 14 17:05:13.663408 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.663328    2580 manager.go:354] Recovery completed
Jul 14 17:05:13.669182 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.669167    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:05:13.671159 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.671135    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:05:13.671241 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.671176    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:05:13.671241 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.671188    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:05:13.671711 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.671692    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 14 17:05:13.671711 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.671710    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 14 17:05:13.671815 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.671726    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 17:05:13.671815 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.671710    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:05:13.672487 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.672472    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:05:13.672544 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.672500    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:05:13.672544 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.672510    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:05:13.672980 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.672969    2580 policy_none.go:49] "None policy: Start"
Jul 14 17:05:13.673015 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.672985    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 14 17:05:13.673015 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.672995    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 14 17:05:13.673726 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.673657    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-134.ec2.internal.18c236da505cb418  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-134.ec2.internal,UID:ip-10-0-137-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-134.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-134.ec2.internal,},FirstTimestamp:2026-07-14 17:05:13.671160856 +0000 UTC m=+0.418760730,LastTimestamp:2026-07-14 17:05:13.671160856 +0000 UTC m=+0.418760730,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-134.ec2.internal,}"
Jul 14 17:05:13.673941 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.673928    2580 policy_none.go:47] "Start"
Jul 14 17:05:13.705357 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.705337    2580 manager.go:337] "Starting Device Plugin manager"
Jul 14 17:05:13.705454 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.705413    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 14 17:05:13.705454 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.705428    2580 server.go:87] "Starting device plugin registration server"
Jul 14 17:05:13.705698 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.705683    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 14 17:05:13.705760 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.705700    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 14 17:05:13.705803 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.705781    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 14 17:05:13.705937 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.705888    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 14 17:05:13.705937 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.705901    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 14 17:05:13.706773 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.706751    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 14 17:05:13.706890 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.706811    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:13.806114 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.806064    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:05:13.807373 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.807356    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:05:13.807441 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.807388    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:05:13.807441 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.807400    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:05:13.807441 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.807427    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-134.ec2.internal"
Jul 14 17:05:13.816662 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.816610    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-134.ec2.internal"
Jul 14 17:05:13.816662 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.816640    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-134.ec2.internal\": node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:13.836573 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.836543    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:13.875219 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.875186    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 14 17:05:13.875345 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.875231    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 14 17:05:13.875345 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.875254    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 14 17:05:13.875345 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.875331    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 14 17:05:13.877287 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.877268    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 17:05:13.937496 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:13.937453    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:13.976036 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.976004    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal"]
Jul 14 17:05:13.976134 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.976082    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:05:13.977117 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.977100    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:05:13.977236 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.977134    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:05:13.977236 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.977147    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:05:13.978499 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.978482    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:05:13.978620 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.978597    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:13.978672 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.978633    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:05:13.979200 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.979184    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:05:13.979276 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.979215    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:05:13.979276 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.979219    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:05:13.979276 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.979237    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:05:13.979276 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.979249    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:05:13.979276 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.979230    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:05:13.980487 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.980471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:13.980562 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.980500    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:05:13.981178 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.981163    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:05:13.981238 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.981190    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:05:13.981238 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:13.981200    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:05:14.005545 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:14.005522    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-134.ec2.internal\" not found" node="ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.010017 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:14.010000    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-134.ec2.internal\" not found" node="ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.037685 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:14.037662    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:14.055098 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.055070    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/319818a0820800557405ab012a22c360-config\") pod \"kube-apiserver-proxy-ip-10-0-137-134.ec2.internal\" (UID: \"319818a0820800557405ab012a22c360\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.055186 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.055101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a289416899e7b5aa8194e0ce6e4e143-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal\" (UID: \"5a289416899e7b5aa8194e0ce6e4e143\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.055186 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.055125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a289416899e7b5aa8194e0ce6e4e143-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal\" (UID: \"5a289416899e7b5aa8194e0ce6e4e143\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.138319 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:14.138248    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:14.155640 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.155613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/319818a0820800557405ab012a22c360-config\") pod \"kube-apiserver-proxy-ip-10-0-137-134.ec2.internal\" (UID: \"319818a0820800557405ab012a22c360\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.155712 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.155645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a289416899e7b5aa8194e0ce6e4e143-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal\" (UID: \"5a289416899e7b5aa8194e0ce6e4e143\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.155712 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.155663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a289416899e7b5aa8194e0ce6e4e143-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal\" (UID: \"5a289416899e7b5aa8194e0ce6e4e143\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.155712 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.155703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a289416899e7b5aa8194e0ce6e4e143-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal\" (UID: \"5a289416899e7b5aa8194e0ce6e4e143\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.155814 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.155721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a289416899e7b5aa8194e0ce6e4e143-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal\" (UID: \"5a289416899e7b5aa8194e0ce6e4e143\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.155814 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.155703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/319818a0820800557405ab012a22c360-config\") pod \"kube-apiserver-proxy-ip-10-0-137-134.ec2.internal\" (UID: \"319818a0820800557405ab012a22c360\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.238818 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:14.238784    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:14.307240 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.307211    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.313488 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.313467    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.339810 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:14.339775    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:14.440268 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:14.440177    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:14.540628 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:14.540583    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:14.588098 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.588071    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 14 17:05:14.588660 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.588233    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 17:05:14.641611 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:14.641574    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:14.664936 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.664885    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 17:00:13 +0000 UTC" deadline="2028-04-29 22:49:57.804632794 +0000 UTC"
Jul 14 17:05:14.664936 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.664919    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15725h44m43.139716678s"
Jul 14 17:05:14.742468 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:14.742402    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-134.ec2.internal\" not found"
Jul 14 17:05:14.837259 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.837232    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 17:05:14.851390 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.851367    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.861770 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.861745    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 17:05:14.864451 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.864432    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal"
Jul 14 17:05:14.875382 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:14.875365    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 17:05:15.035076 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:15.035042    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod319818a0820800557405ab012a22c360.slice/crio-8343e264b51a4b8834f637b37b80af1497e2ec25a124502be875a5b82f5b147b WatchSource:0}: Error finding container 8343e264b51a4b8834f637b37b80af1497e2ec25a124502be875a5b82f5b147b: Status 404 returned error can't find the container with id 8343e264b51a4b8834f637b37b80af1497e2ec25a124502be875a5b82f5b147b
Jul 14 17:05:15.035523 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.035503    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 17:05:15.036305 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:15.036291    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a289416899e7b5aa8194e0ce6e4e143.slice/crio-9306a7c39fa2ac75f247d6cbab9f9f8995737c367fa4fb3b80701fed89d2af59 WatchSource:0}: Error finding container 9306a7c39fa2ac75f247d6cbab9f9f8995737c367fa4fb3b80701fed89d2af59: Status 404 returned error can't find the container with id 9306a7c39fa2ac75f247d6cbab9f9f8995737c367fa4fb3b80701fed89d2af59
Jul 14 17:05:15.042327 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.042308    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 17:05:15.137376 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.137347    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 17:05:15.632695 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.632665    2580 apiserver.go:52] "Watching apiserver"
Jul 14 17:05:15.640777 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.640746    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 14 17:05:15.641505 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.641145    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-t8zcm","kube-system/konnectivity-agent-x28gh","openshift-image-registry/node-ca-4ksph","openshift-multus/multus-additional-cni-plugins-ng8k2","openshift-network-operator/iptables-alerter-5h45q","openshift-ovn-kubernetes/ovnkube-node-r59wt","kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv","openshift-cluster-node-tuning-operator/tuned-4jmxx","openshift-dns/node-resolver-8wp2t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal","openshift-multus/multus-ccqnb","openshift-multus/network-metrics-daemon-zg6rn","openshift-network-diagnostics/network-check-target-plxbp"]
Jul 14 17:05:15.642705 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.642563    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.645658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.645731    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.646150    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.646364    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xs6gm\""
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.646598    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.647063    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.650174    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.650367    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.650538    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lsrh4\""
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.650732    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.651031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.651946    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.653927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.652700    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.654677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.654015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.654677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.654041    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 14 17:05:15.654677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.654049    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 14 17:05:15.654677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.654115    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cdmfg\""
Jul 14 17:05:15.654677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.654241    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:15.654677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.654290    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 14 17:05:15.654677 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.654311    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:15.654677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.654412    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 14 17:05:15.654677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.654527    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 14 17:05:15.654677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.654594    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:15.655197 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.655082    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:15.655197 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.655137    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:15.655592 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.655447    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 14 17:05:15.657978 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.657951    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 14 17:05:15.658163 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.658149    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 14 17:05:15.658339 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.658324    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 14 17:05:15.658523 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.658509    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xjg44\""
Jul 14 17:05:15.658699 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.658683    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2zr5h\""
Jul 14 17:05:15.659204 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.659187    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 14 17:05:15.659411 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.659378    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 14 17:05:15.659496 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.659484    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 14 17:05:15.659496 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.659516    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 14 17:05:15.659618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.659579    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 14 17:05:15.659686 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.659669    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gdjp2\""
Jul 14 17:05:15.660607 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.659958    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 14 17:05:15.660607 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.660049    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bvg2x\""
Jul 14 17:05:15.660607 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.660052    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 14 17:05:15.660607 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.660112    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 14 17:05:15.660607 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.660194    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 14 17:05:15.660607 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.660414    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 14 17:05:15.662739 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.662719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.662856 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.662744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-kubernetes\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.662856 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.662797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-sysctl-d\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.662856 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.662826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e-hosts-file\") pod \"node-resolver-8wp2t\" (UID: \"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e\") " pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.663031 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.662877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5bxn\" (UniqueName: \"kubernetes.io/projected/f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e-kube-api-access-b5bxn\") pod \"node-resolver-8wp2t\" (UID: \"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e\") " pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.663031 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.662904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4vt7\" (UniqueName: \"kubernetes.io/projected/9ea08ac1-2015-4bf9-b169-8b93b02530d3-kube-api-access-m4vt7\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.663031 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.662935    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-system-cni-dir\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.663031 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.662962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh6zs\" (UniqueName: \"kubernetes.io/projected/e1f52c4b-6f9f-4351-b020-bdd00ed53d5e-kube-api-access-xh6zs\") pod \"iptables-alerter-5h45q\" (UID: \"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e\") " pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.663031 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.662989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-run-ovn\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.663031 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663018    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-modprobe-d\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.663305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-sysctl-conf\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.663305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-os-release\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.663305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46d1e4d2-c328-4d86-a524-0b64bbc9fdf0-agent-certs\") pod \"konnectivity-agent-x28gh\" (UID: \"46d1e4d2-c328-4d86-a524-0b64bbc9fdf0\") " pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:15.663305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-log-socket\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.663305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663147    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b90c2ddb-a4ec-4f83-8a92-6f629b633810-env-overrides\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.663305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlt67\" (UniqueName: \"kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67\") pod \"network-check-target-plxbp\" (UID: \"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8\") " pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:15.663305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663198    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz5dd\" (UniqueName: \"kubernetes.io/projected/4cefdd01-3531-4e69-89da-a20d63e8f853-kube-api-access-cz5dd\") pod \"node-ca-4ksph\" (UID: \"4cefdd01-3531-4e69-89da-a20d63e8f853\") " pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.663305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-cni-bin\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.663305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsg6l\" (UniqueName: \"kubernetes.io/projected/b90c2ddb-a4ec-4f83-8a92-6f629b633810-kube-api-access-rsg6l\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.663305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-lib-modules\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-sysconfig\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4cefdd01-3531-4e69-89da-a20d63e8f853-host\") pod \"node-ca-4ksph\" (UID: \"4cefdd01-3531-4e69-89da-a20d63e8f853\") " pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e-tmp-dir\") pod \"node-resolver-8wp2t\" (UID: \"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e\") " pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e1f52c4b-6f9f-4351-b020-bdd00ed53d5e-iptables-alerter-script\") pod \"iptables-alerter-5h45q\" (UID: \"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e\") " pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0041d907-ca0f-4192-b156-70cf568ec6df-etc-tuned\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0041d907-ca0f-4192-b156-70cf568ec6df-tmp\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663562    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-run-openvswitch\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-cni-netd\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-sys\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663669    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-dbus\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:15.663731 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.663687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-slash\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.664372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-run-ovn-kubernetes\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.664372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:15.664372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664164    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-kubelet\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.664372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-run-netns\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.664372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4cefdd01-3531-4e69-89da-a20d63e8f853-serviceca\") pod \"node-ca-4ksph\" (UID: \"4cefdd01-3531-4e69-89da-a20d63e8f853\") " pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.664372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-run-systemd\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.664372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-run\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.664372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-kubelet-config\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:15.664372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-var-lib-openvswitch\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.664776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b90c2ddb-a4ec-4f83-8a92-6f629b633810-ovn-node-metrics-cert\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.664776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-systemd\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.664776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ea08ac1-2015-4bf9-b169-8b93b02530d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.664776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-etc-openvswitch\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.664776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:15.664776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b296r\" (UniqueName: \"kubernetes.io/projected/b40f2043-fe66-46e3-8d61-2567218a4c1e-kube-api-access-b296r\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:15.664776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664580    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46d1e4d2-c328-4d86-a524-0b64bbc9fdf0-konnectivity-ca\") pod \"konnectivity-agent-x28gh\" (UID: \"46d1e4d2-c328-4d86-a524-0b64bbc9fdf0\") " pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:15.664776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-var-lib-kubelet\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.664776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-host\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.664776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gd4z\" (UniqueName: \"kubernetes.io/projected/0041d907-ca0f-4192-b156-70cf568ec6df-kube-api-access-6gd4z\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.665310 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-cnibin\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.665310 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.665310 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-systemd-units\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.665310 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-node-log\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.665310 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ea08ac1-2015-4bf9-b169-8b93b02530d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.665310 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ea08ac1-2015-4bf9-b169-8b93b02530d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.665310 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.664996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1f52c4b-6f9f-4351-b020-bdd00ed53d5e-host-slash\") pod \"iptables-alerter-5h45q\" (UID: \"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e\") " pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.665310 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.665024    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b90c2ddb-a4ec-4f83-8a92-6f629b633810-ovnkube-config\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.665310 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.665049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b90c2ddb-a4ec-4f83-8a92-6f629b633810-ovnkube-script-lib\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.666008 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.665425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.666008 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.665886    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 14 17:05:15.666008 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.665976    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 14 17:05:15.666197 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.666052    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 14 17:05:15.666197 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.665888    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jvm84\""
Jul 14 17:05:15.667784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.667755    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5b8xk\""
Jul 14 17:05:15.668063 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.668044    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 14 17:05:15.673892 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.673723    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 17:05:15.690970 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.690935    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ccq8j"
Jul 14 17:05:15.700623 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.700604    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ccq8j"
Jul 14 17:05:15.753271 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.753243    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 14 17:05:15.765713 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.765681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-etc-openvswitch\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.765902 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.765721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-device-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.765902 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.765742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-os-release\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.765902 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.765789    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-socket-dir-parent\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.765902 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.765803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-etc-openvswitch\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.765902 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.765881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:15.766198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.765915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b296r\" (UniqueName: \"kubernetes.io/projected/b40f2043-fe66-46e3-8d61-2567218a4c1e-kube-api-access-b296r\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:15.766198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.765941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46d1e4d2-c328-4d86-a524-0b64bbc9fdf0-konnectivity-ca\") pod \"konnectivity-agent-x28gh\" (UID: \"46d1e4d2-c328-4d86-a524-0b64bbc9fdf0\") " pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:15.766198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.765968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-var-lib-kubelet\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.766198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.765999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-host\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.766198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gd4z\" (UniqueName: \"kubernetes.io/projected/0041d907-ca0f-4192-b156-70cf568ec6df-kube-api-access-6gd4z\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.766198 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.766037    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:15.766198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-var-lib-kubelet\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.766198 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.766107    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs podName:b40f2043-fe66-46e3-8d61-2567218a4c1e nodeName:}" failed. No retries permitted until 2026-07-14 17:05:16.266084947 +0000 UTC m=+3.013684815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs") pod "network-metrics-daemon-zg6rn" (UID: "b40f2043-fe66-46e3-8d61-2567218a4c1e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:15.766198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-cnibin\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.766198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-systemd-units\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-host\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-node-log\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ea08ac1-2015-4bf9-b169-8b93b02530d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-cnibin\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ea08ac1-2015-4bf9-b169-8b93b02530d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46d1e4d2-c328-4d86-a524-0b64bbc9fdf0-konnectivity-ca\") pod \"konnectivity-agent-x28gh\" (UID: \"46d1e4d2-c328-4d86-a524-0b64bbc9fdf0\") " pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1f52c4b-6f9f-4351-b020-bdd00ed53d5e-host-slash\") pod \"iptables-alerter-5h45q\" (UID: \"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e\") " pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b90c2ddb-a4ec-4f83-8a92-6f629b633810-ovnkube-config\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b90c2ddb-a4ec-4f83-8a92-6f629b633810-ovnkube-script-lib\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766620    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1f52c4b-6f9f-4351-b020-bdd00ed53d5e-host-slash\") pod \"iptables-alerter-5h45q\" (UID: \"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e\") " pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-node-log\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766648    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-socket-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-run-netns\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.766670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ea08ac1-2015-4bf9-b169-8b93b02530d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766450    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-systemd-units\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-kubernetes\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-kubernetes\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-sysctl-d\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e-hosts-file\") pod \"node-resolver-8wp2t\" (UID: \"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e\") " pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bxn\" (UniqueName: \"kubernetes.io/projected/f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e-kube-api-access-b5bxn\") pod \"node-resolver-8wp2t\" (UID: \"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e\") " pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4vt7\" (UniqueName: \"kubernetes.io/projected/9ea08ac1-2015-4bf9-b169-8b93b02530d3-kube-api-access-m4vt7\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-etc-selinux\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfzfk\" (UniqueName: \"kubernetes.io/projected/29dec697-3fcd-46bf-ba73-1e51e7f58e93-kube-api-access-dfzfk\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-system-cni-dir\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ea08ac1-2015-4bf9-b169-8b93b02530d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.766956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh6zs\" (UniqueName: \"kubernetes.io/projected/e1f52c4b-6f9f-4351-b020-bdd00ed53d5e-kube-api-access-xh6zs\") pod \"iptables-alerter-5h45q\" (UID: \"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e\") " pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b90c2ddb-a4ec-4f83-8a92-6f629b633810-ovnkube-config\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e-hosts-file\") pod \"node-resolver-8wp2t\" (UID: \"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e\") " pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-run-ovn\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767131    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b90c2ddb-a4ec-4f83-8a92-6f629b633810-ovnkube-script-lib\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.767481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-modprobe-d\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-system-cni-dir\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-sysctl-conf\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767193    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-sysctl-d\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767209    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-run-ovn\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-os-release\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46d1e4d2-c328-4d86-a524-0b64bbc9fdf0-agent-certs\") pod \"konnectivity-agent-x28gh\" (UID: \"46d1e4d2-c328-4d86-a524-0b64bbc9fdf0\") " pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-modprobe-d\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-log-socket\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b90c2ddb-a4ec-4f83-8a92-6f629b633810-env-overrides\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ea08ac1-2015-4bf9-b169-8b93b02530d3-os-release\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-conf-dir\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-log-socket\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-sysctl-conf\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlt67\" (UniqueName: \"kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67\") pod \"network-check-target-plxbp\" (UID: \"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8\") " pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz5dd\" (UniqueName: \"kubernetes.io/projected/4cefdd01-3531-4e69-89da-a20d63e8f853-kube-api-access-cz5dd\") pod \"node-ca-4ksph\" (UID: \"4cefdd01-3531-4e69-89da-a20d63e8f853\") " pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-cni-bin\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.768282 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsg6l\" (UniqueName: \"kubernetes.io/projected/b90c2ddb-a4ec-4f83-8a92-6f629b633810-kube-api-access-rsg6l\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767442    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-daemon-config\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-lib-modules\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-sysconfig\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4cefdd01-3531-4e69-89da-a20d63e8f853-host\") pod \"node-ca-4ksph\" (UID: \"4cefdd01-3531-4e69-89da-a20d63e8f853\") " pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767540    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-registration-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e-tmp-dir\") pod \"node-resolver-8wp2t\" (UID: \"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e\") " pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767596    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b90c2ddb-a4ec-4f83-8a92-6f629b633810-env-overrides\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e1f52c4b-6f9f-4351-b020-bdd00ed53d5e-iptables-alerter-script\") pod \"iptables-alerter-5h45q\" (UID: \"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e\") " pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767631    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-cni-bin\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-sys-fs\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/896c036a-a2ef-4f00-9af9-3c957ecba296-cni-binary-copy\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-sysconfig\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-run-k8s-cni-cncf-io\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0041d907-ca0f-4192-b156-70cf568ec6df-etc-tuned\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-lib-modules\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.767749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0041d907-ca0f-4192-b156-70cf568ec6df-tmp\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.769025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e1f52c4b-6f9f-4351-b020-bdd00ed53d5e-iptables-alerter-script\") pod \"iptables-alerter-5h45q\" (UID: \"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e\") " pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-run-openvswitch\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-cni-netd\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768407    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4cefdd01-3531-4e69-89da-a20d63e8f853-host\") pod \"node-ca-4ksph\" (UID: \"4cefdd01-3531-4e69-89da-a20d63e8f853\") " pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-system-cni-dir\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768467    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-run-openvswitch\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-var-lib-cni-multus\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-run-multus-certs\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768561    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-cni-netd\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-sys\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e-tmp-dir\") pod \"node-resolver-8wp2t\" (UID: \"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e\") " pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-dbus\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-cni-dir\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-cnibin\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-dbus\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:15.769753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768810    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6qcr\" (UniqueName: \"kubernetes.io/projected/896c036a-a2ef-4f00-9af9-3c957ecba296-kube-api-access-w6qcr\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-sys\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-slash\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-run-ovn-kubernetes\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-hostroot\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-slash\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.768982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-etc-kubernetes\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-run-ovn-kubernetes\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-kubelet\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-run-netns\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.769102    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.769152    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret podName:ccda64ad-05c5-4aff-95f2-ea99d638c5aa nodeName:}" failed. No retries permitted until 2026-07-14 17:05:16.269136041 +0000 UTC m=+3.016735912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret") pod "global-pull-secret-syncer-t8zcm" (UID: "ccda64ad-05c5-4aff-95f2-ea99d638c5aa") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4cefdd01-3531-4e69-89da-a20d63e8f853-serviceca\") pod \"node-ca-4ksph\" (UID: \"4cefdd01-3531-4e69-89da-a20d63e8f853\") " pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-run-systemd\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-run\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-kubelet-config\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:15.770618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769268    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-run-systemd\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-var-lib-openvswitch\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-var-lib-openvswitch\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-run\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-run-netns\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-kubelet-config\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b90c2ddb-a4ec-4f83-8a92-6f629b633810-host-kubelet\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b90c2ddb-a4ec-4f83-8a92-6f629b633810-ovn-node-metrics-cert\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-var-lib-cni-bin\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769669    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-var-lib-kubelet\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-systemd\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ea08ac1-2015-4bf9-b169-8b93b02530d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.769857    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0041d907-ca0f-4192-b156-70cf568ec6df-etc-systemd\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.770177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4cefdd01-3531-4e69-89da-a20d63e8f853-serviceca\") pod \"node-ca-4ksph\" (UID: \"4cefdd01-3531-4e69-89da-a20d63e8f853\") " pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.770334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ea08ac1-2015-4bf9-b169-8b93b02530d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.771443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.771395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46d1e4d2-c328-4d86-a524-0b64bbc9fdf0-agent-certs\") pod \"konnectivity-agent-x28gh\" (UID: \"46d1e4d2-c328-4d86-a524-0b64bbc9fdf0\") " pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:15.772117 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.772099    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0041d907-ca0f-4192-b156-70cf568ec6df-etc-tuned\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.772171 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.772141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0041d907-ca0f-4192-b156-70cf568ec6df-tmp\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.773878 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.773767    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b90c2ddb-a4ec-4f83-8a92-6f629b633810-ovn-node-metrics-cert\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.777636 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.777612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gd4z\" (UniqueName: \"kubernetes.io/projected/0041d907-ca0f-4192-b156-70cf568ec6df-kube-api-access-6gd4z\") pod \"tuned-4jmxx\" (UID: \"0041d907-ca0f-4192-b156-70cf568ec6df\") " pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:15.778064 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.778043    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:05:15.778158 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.778068    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:05:15.778158 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.778081    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rlt67 for pod openshift-network-diagnostics/network-check-target-plxbp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:15.778158 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:15.778136    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67 podName:7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:16.278119275 +0000 UTC m=+3.025719140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rlt67" (UniqueName: "kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67") pod "network-check-target-plxbp" (UID: "7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:15.778288 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.778160    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b296r\" (UniqueName: \"kubernetes.io/projected/b40f2043-fe66-46e3-8d61-2567218a4c1e-kube-api-access-b296r\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:15.780459 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.780437    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5bxn\" (UniqueName: \"kubernetes.io/projected/f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e-kube-api-access-b5bxn\") pod \"node-resolver-8wp2t\" (UID: \"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e\") " pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.780568 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.780466    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4vt7\" (UniqueName: \"kubernetes.io/projected/9ea08ac1-2015-4bf9-b169-8b93b02530d3-kube-api-access-m4vt7\") pod \"multus-additional-cni-plugins-ng8k2\" (UID: \"9ea08ac1-2015-4bf9-b169-8b93b02530d3\") " pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.780649 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.780631    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz5dd\" (UniqueName: \"kubernetes.io/projected/4cefdd01-3531-4e69-89da-a20d63e8f853-kube-api-access-cz5dd\") pod \"node-ca-4ksph\" (UID: \"4cefdd01-3531-4e69-89da-a20d63e8f853\") " pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.782150 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.782128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh6zs\" (UniqueName: \"kubernetes.io/projected/e1f52c4b-6f9f-4351-b020-bdd00ed53d5e-kube-api-access-xh6zs\") pod \"iptables-alerter-5h45q\" (UID: \"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e\") " pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.782463 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.782443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsg6l\" (UniqueName: \"kubernetes.io/projected/b90c2ddb-a4ec-4f83-8a92-6f629b633810-kube-api-access-rsg6l\") pod \"ovnkube-node-r59wt\" (UID: \"b90c2ddb-a4ec-4f83-8a92-6f629b633810\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:15.871172 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-system-cni-dir\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871332 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-var-lib-cni-multus\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871332 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-run-multus-certs\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871332 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-cni-dir\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871332 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-cnibin\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871332 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6qcr\" (UniqueName: \"kubernetes.io/projected/896c036a-a2ef-4f00-9af9-3c957ecba296-kube-api-access-w6qcr\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871332 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-hostroot\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-etc-kubernetes\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-var-lib-cni-bin\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-var-lib-kubelet\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871439    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-device-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.871604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-os-release\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-socket-dir-parent\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.871604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-socket-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.871604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-run-netns\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-etc-selinux\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.871969 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfzfk\" (UniqueName: \"kubernetes.io/projected/29dec697-3fcd-46bf-ba73-1e51e7f58e93-kube-api-access-dfzfk\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.871969 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-conf-dir\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871969 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-daemon-config\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871969 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-registration-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.871969 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-sys-fs\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.871969 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/896c036a-a2ef-4f00-9af9-3c957ecba296-cni-binary-copy\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871969 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-run-k8s-cni-cncf-io\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871969 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-run-k8s-cni-cncf-io\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871969 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-system-cni-dir\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.871969 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871957    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-var-lib-cni-multus\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872358 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.871982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-run-multus-certs\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872358 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-cni-dir\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872358 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-cnibin\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872358 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872344    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-hostroot\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872545 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872397    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-etc-kubernetes\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872545 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-var-lib-cni-bin\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872545 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872484    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-var-lib-kubelet\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872545 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-device-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.872697 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-os-release\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872697 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-socket-dir-parent\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872780 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872706    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.872872 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-socket-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.872924 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-host-run-netns\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.872982 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.872966    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-etc-selinux\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.873274 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.873245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-conf-dir\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.873813 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.873786    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/896c036a-a2ef-4f00-9af9-3c957ecba296-multus-daemon-config\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.873957 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.873888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-registration-dir\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.873957 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.873942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29dec697-3fcd-46bf-ba73-1e51e7f58e93-sys-fs\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.874335 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.874315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/896c036a-a2ef-4f00-9af9-3c957ecba296-cni-binary-copy\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.886285 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.886094    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal" event={"ID":"5a289416899e7b5aa8194e0ce6e4e143","Type":"ContainerStarted","Data":"9306a7c39fa2ac75f247d6cbab9f9f8995737c367fa4fb3b80701fed89d2af59"}
Jul 14 17:05:15.888554 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.888528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfzfk\" (UniqueName: \"kubernetes.io/projected/29dec697-3fcd-46bf-ba73-1e51e7f58e93-kube-api-access-dfzfk\") pod \"aws-ebs-csi-driver-node-t2dbv\" (UID: \"29dec697-3fcd-46bf-ba73-1e51e7f58e93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:15.890646 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.890587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal" event={"ID":"319818a0820800557405ab012a22c360","Type":"ContainerStarted","Data":"8343e264b51a4b8834f637b37b80af1497e2ec25a124502be875a5b82f5b147b"}
Jul 14 17:05:15.892803 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.892777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6qcr\" (UniqueName: \"kubernetes.io/projected/896c036a-a2ef-4f00-9af9-3c957ecba296-kube-api-access-w6qcr\") pod \"multus-ccqnb\" (UID: \"896c036a-a2ef-4f00-9af9-3c957ecba296\") " pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:15.970347 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.970308    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8wp2t"
Jul 14 17:05:15.979902 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.979873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4ksph"
Jul 14 17:05:15.986719 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.986691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ng8k2"
Jul 14 17:05:15.993801 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.993779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5h45q"
Jul 14 17:05:15.994050 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:15.994031    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 17:05:16.002227 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.002203    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:16.009958 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.009937    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4jmxx"
Jul 14 17:05:16.017670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.017646    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:16.023371 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.023351    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv"
Jul 14 17:05:16.029918 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.029899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ccqnb"
Jul 14 17:05:16.274662 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.274580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:16.274662 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.274619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:16.274905 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.274722    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:16.274905 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.274734    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:16.274905 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.274775    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs podName:b40f2043-fe66-46e3-8d61-2567218a4c1e nodeName:}" failed. No retries permitted until 2026-07-14 17:05:17.274758795 +0000 UTC m=+4.022358667 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs") pod "network-metrics-daemon-zg6rn" (UID: "b40f2043-fe66-46e3-8d61-2567218a4c1e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:16.274905 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.274802    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret podName:ccda64ad-05c5-4aff-95f2-ea99d638c5aa nodeName:}" failed. No retries permitted until 2026-07-14 17:05:17.274782598 +0000 UTC m=+4.022382457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret") pod "global-pull-secret-syncer-t8zcm" (UID: "ccda64ad-05c5-4aff-95f2-ea99d638c5aa") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:16.375052 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.375016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlt67\" (UniqueName: \"kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67\") pod \"network-check-target-plxbp\" (UID: \"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8\") " pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:16.375219 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.375143    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:05:16.375219 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.375172    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:05:16.375219 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.375184    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rlt67 for pod openshift-network-diagnostics/network-check-target-plxbp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:16.375351 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.375241    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67 podName:7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:17.375227279 +0000 UTC m=+4.122827134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rlt67" (UniqueName: "kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67") pod "network-check-target-plxbp" (UID: "7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:16.702743 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.702655    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:00:15 +0000 UTC" deadline="2028-02-13 15:09:15.622665396 +0000 UTC"
Jul 14 17:05:16.702743 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.702691    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13894h3m58.919977951s"
Jul 14 17:05:16.875740 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.875703    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:16.875959 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.875703    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:16.875959 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.875865    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:16.876078 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.875959    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:16.876078 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:16.875717    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:16.876078 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:16.876068    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:16.952957 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:16.952877    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb90c2ddb_a4ec_4f83_8a92_6f629b633810.slice/crio-9dd84d1afb37a5c2c92ba13d6b5e02eee5937d051aeb09e882144fa78c84fc4d WatchSource:0}: Error finding container 9dd84d1afb37a5c2c92ba13d6b5e02eee5937d051aeb09e882144fa78c84fc4d: Status 404 returned error can't find the container with id 9dd84d1afb37a5c2c92ba13d6b5e02eee5937d051aeb09e882144fa78c84fc4d
Jul 14 17:05:16.955069 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:16.955040    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29dec697_3fcd_46bf_ba73_1e51e7f58e93.slice/crio-d7d87509f80b6d97c3232423fb140cef068e379ef154edec8d4da51c53499399 WatchSource:0}: Error finding container d7d87509f80b6d97c3232423fb140cef068e379ef154edec8d4da51c53499399: Status 404 returned error can't find the container with id d7d87509f80b6d97c3232423fb140cef068e379ef154edec8d4da51c53499399
Jul 14 17:05:16.960558 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:16.960533    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0041d907_ca0f_4192_b156_70cf568ec6df.slice/crio-a3a87e8f6c80ef80182cc74292e64e18a7b0a2fbbafe734a31884de1409a6ef5 WatchSource:0}: Error finding container a3a87e8f6c80ef80182cc74292e64e18a7b0a2fbbafe734a31884de1409a6ef5: Status 404 returned error can't find the container with id a3a87e8f6c80ef80182cc74292e64e18a7b0a2fbbafe734a31884de1409a6ef5
Jul 14 17:05:16.961383 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:16.961359    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1bb8675_6bf3_4e71_b94d_8d3f0af7f88e.slice/crio-0db5baae1a7a52e9896ff96e7f97bc517621c414e33dc5d2bfb7ba2336fd10d5 WatchSource:0}: Error finding container 0db5baae1a7a52e9896ff96e7f97bc517621c414e33dc5d2bfb7ba2336fd10d5: Status 404 returned error can't find the container with id 0db5baae1a7a52e9896ff96e7f97bc517621c414e33dc5d2bfb7ba2336fd10d5
Jul 14 17:05:16.963301 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:16.962941    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46d1e4d2_c328_4d86_a524_0b64bbc9fdf0.slice/crio-625d31675b29f04e2ce5da31842a6712660c76d99f4386371ed77d6384826791 WatchSource:0}: Error finding container 625d31675b29f04e2ce5da31842a6712660c76d99f4386371ed77d6384826791: Status 404 returned error can't find the container with id 625d31675b29f04e2ce5da31842a6712660c76d99f4386371ed77d6384826791
Jul 14 17:05:16.963690 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:16.963666    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f52c4b_6f9f_4351_b020_bdd00ed53d5e.slice/crio-b2ad9303cca788bae31fa4c5953e658150d560e7fde39f547e3a7a96036a7eea WatchSource:0}: Error finding container b2ad9303cca788bae31fa4c5953e658150d560e7fde39f547e3a7a96036a7eea: Status 404 returned error can't find the container with id b2ad9303cca788bae31fa4c5953e658150d560e7fde39f547e3a7a96036a7eea
Jul 14 17:05:16.966706 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:16.966660    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cefdd01_3531_4e69_89da_a20d63e8f853.slice/crio-d448eb1a26c8443cbd4712f29254dfd314ab3f3cb8426fbe7d04692d2a6eb2b3 WatchSource:0}: Error finding container d448eb1a26c8443cbd4712f29254dfd314ab3f3cb8426fbe7d04692d2a6eb2b3: Status 404 returned error can't find the container with id d448eb1a26c8443cbd4712f29254dfd314ab3f3cb8426fbe7d04692d2a6eb2b3
Jul 14 17:05:17.280376 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.280281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:17.280376 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.280352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:17.280571 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:17.280427    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:17.280571 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:17.280473    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:17.280571 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:17.280505    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret podName:ccda64ad-05c5-4aff-95f2-ea99d638c5aa nodeName:}" failed. No retries permitted until 2026-07-14 17:05:19.28048517 +0000 UTC m=+6.028085038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret") pod "global-pull-secret-syncer-t8zcm" (UID: "ccda64ad-05c5-4aff-95f2-ea99d638c5aa") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:17.280571 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:17.280530    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs podName:b40f2043-fe66-46e3-8d61-2567218a4c1e nodeName:}" failed. No retries permitted until 2026-07-14 17:05:19.280512566 +0000 UTC m=+6.028112423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs") pod "network-metrics-daemon-zg6rn" (UID: "b40f2043-fe66-46e3-8d61-2567218a4c1e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:17.381146 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.381104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlt67\" (UniqueName: \"kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67\") pod \"network-check-target-plxbp\" (UID: \"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8\") " pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:17.381337 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:17.381315    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:05:17.381337 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:17.381336    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:05:17.381434 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:17.381349    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rlt67 for pod openshift-network-diagnostics/network-check-target-plxbp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:17.381434 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:17.381411    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67 podName:7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:19.381392762 +0000 UTC m=+6.128992641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rlt67" (UniqueName: "kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67") pod "network-check-target-plxbp" (UID: "7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:17.703300 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.703186    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:00:15 +0000 UTC" deadline="2028-04-08 02:46:52.015972478 +0000 UTC"
Jul 14 17:05:17.703300 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.703248    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15201h41m34.312728922s"
Jul 14 17:05:17.913106 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.912730    2580 generic.go:356] "Generic (PLEG): container finished" podID="5a289416899e7b5aa8194e0ce6e4e143" containerID="7871b915adc3a9bf7be97a49086b1a228d747f5b75909178f91ef92131dc3c40" exitCode=0
Jul 14 17:05:17.913106 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.912835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal" event={"ID":"5a289416899e7b5aa8194e0ce6e4e143","Type":"ContainerDied","Data":"7871b915adc3a9bf7be97a49086b1a228d747f5b75909178f91ef92131dc3c40"}
Jul 14 17:05:17.917758 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.917268    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5h45q" event={"ID":"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e","Type":"ContainerStarted","Data":"b2ad9303cca788bae31fa4c5953e658150d560e7fde39f547e3a7a96036a7eea"}
Jul 14 17:05:17.927443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.927333    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8wp2t" event={"ID":"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e","Type":"ContainerStarted","Data":"0db5baae1a7a52e9896ff96e7f97bc517621c414e33dc5d2bfb7ba2336fd10d5"}
Jul 14 17:05:17.958020 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.957936    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4jmxx" event={"ID":"0041d907-ca0f-4192-b156-70cf568ec6df","Type":"ContainerStarted","Data":"a3a87e8f6c80ef80182cc74292e64e18a7b0a2fbbafe734a31884de1409a6ef5"}
Jul 14 17:05:17.973406 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.973364    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerStarted","Data":"9dd84d1afb37a5c2c92ba13d6b5e02eee5937d051aeb09e882144fa78c84fc4d"}
Jul 14 17:05:17.986150 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:17.986111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal" event={"ID":"319818a0820800557405ab012a22c360","Type":"ContainerStarted","Data":"f891a9596f11b7711b0362b80caf83d7a208647618d6e28347530fa34230abb8"}
Jul 14 17:05:18.000257 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:18.000173    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4ksph" event={"ID":"4cefdd01-3531-4e69-89da-a20d63e8f853","Type":"ContainerStarted","Data":"d448eb1a26c8443cbd4712f29254dfd314ab3f3cb8426fbe7d04692d2a6eb2b3"}
Jul 14 17:05:18.004297 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:18.002957    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ccqnb" event={"ID":"896c036a-a2ef-4f00-9af9-3c957ecba296","Type":"ContainerStarted","Data":"e74f63bc57c9073fb85e3d698d78699bd25a662e4c146a8abd3ac145a785b58f"}
Jul 14 17:05:18.004297 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:18.003178    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-134.ec2.internal" podStartSLOduration=4.003160726 podStartE2EDuration="4.003160726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:18.003021097 +0000 UTC m=+4.750620999" watchObservedRunningTime="2026-07-14 17:05:18.003160726 +0000 UTC m=+4.750760601"
Jul 14 17:05:18.010331 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:18.010298    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x28gh" event={"ID":"46d1e4d2-c328-4d86-a524-0b64bbc9fdf0","Type":"ContainerStarted","Data":"625d31675b29f04e2ce5da31842a6712660c76d99f4386371ed77d6384826791"}
Jul 14 17:05:18.024620 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:18.024521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ng8k2" event={"ID":"9ea08ac1-2015-4bf9-b169-8b93b02530d3","Type":"ContainerStarted","Data":"a4f439c83cc15ed2487dd267837c3b331c5bf122fcd0dba2dd5dac4546c68ea6"}
Jul 14 17:05:18.033831 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:18.033760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv" event={"ID":"29dec697-3fcd-46bf-ba73-1e51e7f58e93","Type":"ContainerStarted","Data":"d7d87509f80b6d97c3232423fb140cef068e379ef154edec8d4da51c53499399"}
Jul 14 17:05:18.876371 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:18.876331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:18.876917 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:18.876492    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:18.876917 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:18.876903    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:18.877043 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:18.876998    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:18.877097 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:18.877064    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:18.877148 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:18.877131    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:19.069607 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:19.069502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal" event={"ID":"5a289416899e7b5aa8194e0ce6e4e143","Type":"ContainerStarted","Data":"37f46f03bb883d63e917d12bfc857e8d0c16308105e737da6197d1ea54379524"}
Jul 14 17:05:19.297746 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:19.296907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:19.297746 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:19.297009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:19.297746 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:19.297149    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:19.297746 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:19.297213    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret podName:ccda64ad-05c5-4aff-95f2-ea99d638c5aa nodeName:}" failed. No retries permitted until 2026-07-14 17:05:23.297192871 +0000 UTC m=+10.044792731 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret") pod "global-pull-secret-syncer-t8zcm" (UID: "ccda64ad-05c5-4aff-95f2-ea99d638c5aa") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:19.297746 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:19.297617    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:19.297746 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:19.297662    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs podName:b40f2043-fe66-46e3-8d61-2567218a4c1e nodeName:}" failed. No retries permitted until 2026-07-14 17:05:23.297646857 +0000 UTC m=+10.045246715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs") pod "network-metrics-daemon-zg6rn" (UID: "b40f2043-fe66-46e3-8d61-2567218a4c1e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:19.398452 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:19.397834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlt67\" (UniqueName: \"kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67\") pod \"network-check-target-plxbp\" (UID: \"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8\") " pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:19.398452 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:19.398083    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:05:19.398452 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:19.398106    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:05:19.398452 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:19.398116    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rlt67 for pod openshift-network-diagnostics/network-check-target-plxbp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:19.398452 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:19.398161    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67 podName:7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:23.398148416 +0000 UTC m=+10.145748270 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rlt67" (UniqueName: "kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67") pod "network-check-target-plxbp" (UID: "7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:20.875690 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:20.875577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:20.876297 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:20.875712    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:20.876297 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:20.876151    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:20.876297 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:20.876259    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:20.876474 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:20.876313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:20.876474 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:20.876393    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:22.876394 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:22.876356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:22.876872 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:22.876495    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:22.876872 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:22.876558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:22.876872 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:22.876638    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:22.876872 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:22.876687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:22.876872 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:22.876755    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:23.332717 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:23.332626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:23.332717 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:23.332685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:23.332967 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:23.332854    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:23.332967 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:23.332921    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs podName:b40f2043-fe66-46e3-8d61-2567218a4c1e nodeName:}" failed. No retries permitted until 2026-07-14 17:05:31.332902166 +0000 UTC m=+18.080502027 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs") pod "network-metrics-daemon-zg6rn" (UID: "b40f2043-fe66-46e3-8d61-2567218a4c1e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:23.333347 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:23.333326    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:23.333437 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:23.333385    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret podName:ccda64ad-05c5-4aff-95f2-ea99d638c5aa nodeName:}" failed. No retries permitted until 2026-07-14 17:05:31.333369032 +0000 UTC m=+18.080968892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret") pod "global-pull-secret-syncer-t8zcm" (UID: "ccda64ad-05c5-4aff-95f2-ea99d638c5aa") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:23.433788 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:23.433749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlt67\" (UniqueName: \"kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67\") pod \"network-check-target-plxbp\" (UID: \"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8\") " pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:23.434013 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:23.433989    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:05:23.434013 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:23.434012    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:05:23.434013 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:23.434021    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rlt67 for pod openshift-network-diagnostics/network-check-target-plxbp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:23.434239 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:23.434078    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67 podName:7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:31.434058502 +0000 UTC m=+18.181658358 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rlt67" (UniqueName: "kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67") pod "network-check-target-plxbp" (UID: "7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:24.876416 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:24.876143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:24.876940 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:24.876143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:24.876940 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:24.876522    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:24.876940 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:24.876635    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:24.876940 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:24.876165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:24.876940 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:24.876738    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:26.876311 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:26.876269    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:26.876785 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:26.876268    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:26.876785 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:26.876416    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:26.876785 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:26.876268    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:26.876785 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:26.876489    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:26.876785 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:26.876544    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:28.875672 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:28.875629    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:28.876162 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:28.875629    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:28.876162 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:28.875782    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:28.876162 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:28.875629    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:28.876162 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:28.875893    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:28.876162 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:28.875935    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:30.876277 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:30.876242    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:30.876742 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:30.876298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:30.876742 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:30.876298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:30.876742 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:30.876376    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:30.876742 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:30.876475    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:30.876742 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:30.876544    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:31.395990 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:31.395696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:31.395990 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:31.395757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:31.395990 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:31.395860    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:31.395990 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:31.395909    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:31.395990 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:31.395936    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret podName:ccda64ad-05c5-4aff-95f2-ea99d638c5aa nodeName:}" failed. No retries permitted until 2026-07-14 17:05:47.395916045 +0000 UTC m=+34.143515916 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret") pod "global-pull-secret-syncer-t8zcm" (UID: "ccda64ad-05c5-4aff-95f2-ea99d638c5aa") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:31.395990 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:31.395962    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs podName:b40f2043-fe66-46e3-8d61-2567218a4c1e nodeName:}" failed. No retries permitted until 2026-07-14 17:05:47.395945493 +0000 UTC m=+34.143545355 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs") pod "network-metrics-daemon-zg6rn" (UID: "b40f2043-fe66-46e3-8d61-2567218a4c1e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:31.496929 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:31.496894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlt67\" (UniqueName: \"kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67\") pod \"network-check-target-plxbp\" (UID: \"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8\") " pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:31.497132 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:31.497078    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:05:31.497132 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:31.497107    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:05:31.497132 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:31.497121    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rlt67 for pod openshift-network-diagnostics/network-check-target-plxbp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:31.497253 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:31.497178    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67 podName:7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:47.497163224 +0000 UTC m=+34.244763078 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rlt67" (UniqueName: "kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67") pod "network-check-target-plxbp" (UID: "7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:32.876385 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:32.876347    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:32.876385 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:32.876383    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:32.877016 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:32.876348    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:32.877016 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:32.876482    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:32.877016 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:32.876573    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:32.877016 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:32.876658    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:34.876160 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:34.876123    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:34.876576 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:34.876243    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:34.876576 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:34.876125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:34.876576 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:34.876335    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:34.876576 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:34.876124    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:34.876576 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:34.876389    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:36.103167 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.102872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4ksph" event={"ID":"4cefdd01-3531-4e69-89da-a20d63e8f853","Type":"ContainerStarted","Data":"0d55fbe798af75360943c7d1d806ed3381676176b62a839e702ef7cf35e630b8"}
Jul 14 17:05:36.104553 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.104528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ccqnb" event={"ID":"896c036a-a2ef-4f00-9af9-3c957ecba296","Type":"ContainerStarted","Data":"99a5c1d26e2a43051611a521a0fd9c3b35fa35a72620d152b13acf76ca26d98e"}
Jul 14 17:05:36.106000 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.105956    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x28gh" event={"ID":"46d1e4d2-c328-4d86-a524-0b64bbc9fdf0","Type":"ContainerStarted","Data":"eb01526d02887ed75253a59bd89f9d09bd775db3178c404288215a36b001025f"}
Jul 14 17:05:36.107490 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.107467    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ea08ac1-2015-4bf9-b169-8b93b02530d3" containerID="d6c92f5bca80cf0d8b018a57625045e76c2699a858800e50fd9148b2c56f5165" exitCode=0
Jul 14 17:05:36.107557 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.107542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ng8k2" event={"ID":"9ea08ac1-2015-4bf9-b169-8b93b02530d3","Type":"ContainerDied","Data":"d6c92f5bca80cf0d8b018a57625045e76c2699a858800e50fd9148b2c56f5165"}
Jul 14 17:05:36.110247 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.110183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv" event={"ID":"29dec697-3fcd-46bf-ba73-1e51e7f58e93","Type":"ContainerStarted","Data":"f4ddc7557f6e5926d09af168982e89ba5f0c53c4b88b498439d5fc2dd6d202b5"}
Jul 14 17:05:36.112740 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.112720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8wp2t" event={"ID":"f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e","Type":"ContainerStarted","Data":"7311e1d25033b989698909f7221b91174d2fb82c307ba3a2f6fea722a3c5eba9"}
Jul 14 17:05:36.114187 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.114164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4jmxx" event={"ID":"0041d907-ca0f-4192-b156-70cf568ec6df","Type":"ContainerStarted","Data":"967b93268b4a1bc93ae10ce4dcd659a13c33cd2413175fe70e84baddd31a4e51"}
Jul 14 17:05:36.116178 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.116156    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerStarted","Data":"20b31cccb1202d06a09b32a35ab8b16e8b5016918e18ebf75e945189986da8e6"}
Jul 14 17:05:36.116280 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.116232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerStarted","Data":"f1a9e3d2ff7f93623964d927364db7a5a1590df52519c6094ba1fb474df4a04b"}
Jul 14 17:05:36.118763 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.118709    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-134.ec2.internal" podStartSLOduration=22.118691607 podStartE2EDuration="22.118691607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:19.086733749 +0000 UTC m=+5.834333642" watchObservedRunningTime="2026-07-14 17:05:36.118691607 +0000 UTC m=+22.866291485"
Jul 14 17:05:36.119257 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.119204    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4ksph" podStartSLOduration=4.4845556030000004 podStartE2EDuration="23.1191886s" totalImagesPullingTime="18.634632997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:36.118298966 +0000 UTC m=+22.865898842" watchObservedRunningTime="2026-07-14 17:05:36.1191886 +0000 UTC m=+22.866788480"
Jul 14 17:05:36.136739 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.136701    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x28gh" podStartSLOduration=4.495322167 podStartE2EDuration="23.136687286s" totalImagesPullingTime="18.641365119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:36.135936384 +0000 UTC m=+22.883536283" watchObservedRunningTime="2026-07-14 17:05:36.136687286 +0000 UTC m=+22.884287178"
Jul 14 17:05:36.179466 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.179421    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ccqnb" podStartSLOduration=4.497480083 podStartE2EDuration="23.179405442s" totalImagesPullingTime="18.681925359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:36.17904426 +0000 UTC m=+22.926644135" watchObservedRunningTime="2026-07-14 17:05:36.179405442 +0000 UTC m=+22.927005318"
Jul 14 17:05:36.195410 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.195372    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8wp2t" podStartSLOduration=4.564059135 podStartE2EDuration="23.195355908s" totalImagesPullingTime="18.631296773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:36.194931347 +0000 UTC m=+22.942531223" watchObservedRunningTime="2026-07-14 17:05:36.195355908 +0000 UTC m=+22.942955785"
Jul 14 17:05:36.216705 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.216661    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4jmxx" podStartSLOduration=4.584938203 podStartE2EDuration="23.216645208s" totalImagesPullingTime="18.631707005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:36.216383145 +0000 UTC m=+22.963983022" watchObservedRunningTime="2026-07-14 17:05:36.216645208 +0000 UTC m=+22.964245085"
Jul 14 17:05:36.876326 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.876290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:36.876326 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.876329    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:36.876550 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:36.876404    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:36.876550 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:36.876290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:36.876641 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:36.876600    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:36.876692 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:36.876669    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:37.121927 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.121750    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:05:37.123734 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.123414    2580 generic.go:356] "Generic (PLEG): container finished" podID="b90c2ddb-a4ec-4f83-8a92-6f629b633810" containerID="20b31cccb1202d06a09b32a35ab8b16e8b5016918e18ebf75e945189986da8e6" exitCode=1
Jul 14 17:05:37.123734 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.123520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerDied","Data":"20b31cccb1202d06a09b32a35ab8b16e8b5016918e18ebf75e945189986da8e6"}
Jul 14 17:05:37.123734 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.123590    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerStarted","Data":"6b18bca5ad10694720f5e8d0c8a5e607706d4f9e1fa377c4a1a40339d61598a7"}
Jul 14 17:05:37.123734 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.123622    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerStarted","Data":"b48333a69e1f08b7de85bd44542a02fa94b924f993ba80354930fa31004886fb"}
Jul 14 17:05:37.123734 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.123662    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerStarted","Data":"1a44255a0d387e2576d32313b019b90e50943edafa33dafeb76a6781bacf366c"}
Jul 14 17:05:37.123734 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.123697    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerStarted","Data":"ac360a36aa96aa66a31fa065ac900698543e89483d43e899f815c29a09a1b6aa"}
Jul 14 17:05:37.126122 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.126094    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv" event={"ID":"29dec697-3fcd-46bf-ba73-1e51e7f58e93","Type":"ContainerStarted","Data":"9e530560881eba0054ebbb4c49d4f7029281245b27fd820fdeed085a2678562a"}
Jul 14 17:05:37.127732 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.127672    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 14 17:05:37.733764 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.733652    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T17:05:37.127686037Z","UUID":"e16438f6-43a9-47ba-8b09-6c1d96ff2727","Handler":null,"Name":"","Endpoint":""}
Jul 14 17:05:37.735893 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.735870    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 14 17:05:37.736041 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:37.735916    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 14 17:05:38.130304 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:38.130216    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5h45q" event={"ID":"e1f52c4b-6f9f-4351-b020-bdd00ed53d5e","Type":"ContainerStarted","Data":"0cced519888220297a15e1add9bc0e7558759e11775c963856cd7ac3d400a537"}
Jul 14 17:05:38.147562 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:38.147514    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5h45q" podStartSLOduration=6.503348768 podStartE2EDuration="25.14749735s" totalImagesPullingTime="18.644148582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:38.147110798 +0000 UTC m=+24.894710696" watchObservedRunningTime="2026-07-14 17:05:38.14749735 +0000 UTC m=+24.895097258"
Jul 14 17:05:38.875899 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:38.875795    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:38.876035 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:38.875928    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:38.876035 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:38.875938    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:38.876035 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:38.875960    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:38.876122 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:38.876034    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:38.876161 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:38.876147    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:39.135367 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:39.135275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv" event={"ID":"29dec697-3fcd-46bf-ba73-1e51e7f58e93","Type":"ContainerStarted","Data":"f74d555d6fda76071f19a34a619c3a41a50c130076c78108f1bde1e1bf37a6c9"}
Jul 14 17:05:39.139372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:39.139332    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:05:39.139746 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:39.139720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerStarted","Data":"83fc7213369ef33c9a7190711a34bcb8786b998978a6fc454963d2b8f5fbd67c"}
Jul 14 17:05:40.470566 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:40.470532    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:40.471253 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:40.471229    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:40.494583 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:40.494537    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2dbv" podStartSLOduration=6.626125391 podStartE2EDuration="27.494520525s" totalImagesPullingTime="20.868395134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:39.156138742 +0000 UTC m=+25.903738637" watchObservedRunningTime="2026-07-14 17:05:40.494520525 +0000 UTC m=+27.242120401"
Jul 14 17:05:40.876534 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:40.876361    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:40.876716 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:40.876393    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:40.876716 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:40.876630    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:40.876716 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:40.876392    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:40.876716 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:40.876695    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:40.876951 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:40.876757    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:41.147640 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:41.147530    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:05:41.147947 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:41.147917    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerStarted","Data":"046ed6a7f81825c7d4a4f38a0b39837117910b7b8d52aa2f6a1466f6335e9b3b"}
Jul 14 17:05:41.148191 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:41.148167    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:41.148400 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:41.148384    2580 scope.go:117] "RemoveContainer" containerID="20b31cccb1202d06a09b32a35ab8b16e8b5016918e18ebf75e945189986da8e6"
Jul 14 17:05:41.149869 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:41.149829    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ea08ac1-2015-4bf9-b169-8b93b02530d3" containerID="0ee8c5deb7ddb9bcedc2e3e86e78612f4aff393256f0b15c251e3a5e2d6c9f17" exitCode=0
Jul 14 17:05:41.149967 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:41.149889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ng8k2" event={"ID":"9ea08ac1-2015-4bf9-b169-8b93b02530d3","Type":"ContainerDied","Data":"0ee8c5deb7ddb9bcedc2e3e86e78612f4aff393256f0b15c251e3a5e2d6c9f17"}
Jul 14 17:05:41.153495 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:41.151004    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:41.154070 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:41.154047    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x28gh"
Jul 14 17:05:41.166605 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:41.166577    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:42.155254 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:42.155228    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:05:42.155759 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:42.155600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" event={"ID":"b90c2ddb-a4ec-4f83-8a92-6f629b633810","Type":"ContainerStarted","Data":"dd2e454b7be7337cb2d5b45bba646f7980da6fd90365b97a495eb9468fb2ef55"}
Jul 14 17:05:42.155759 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:42.155678    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 14 17:05:42.155936 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:42.155921    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:42.173867 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:42.173832    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:42.235649 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:42.235608    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt" podStartSLOduration=10.30192776 podStartE2EDuration="29.235592073s" totalImagesPullingTime="18.933664313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:42.195790431 +0000 UTC m=+28.943390325" watchObservedRunningTime="2026-07-14 17:05:42.235592073 +0000 UTC m=+28.983191950"
Jul 14 17:05:42.612532 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:42.612443    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:05:42.876410 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:42.876334    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:42.876566 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:42.876427    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:42.876566 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:42.876451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:42.876566 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:42.876532    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:42.876566 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:42.876430    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:42.876732 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:42.876612    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:43.159592 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:43.159498    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ea08ac1-2015-4bf9-b169-8b93b02530d3" containerID="1d9e17a546a2ea6eea08b9b98374cfdb40308dff51266a7b1d86514557fcb5cf" exitCode=0
Jul 14 17:05:43.160157 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:43.159583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ng8k2" event={"ID":"9ea08ac1-2015-4bf9-b169-8b93b02530d3","Type":"ContainerDied","Data":"1d9e17a546a2ea6eea08b9b98374cfdb40308dff51266a7b1d86514557fcb5cf"}
Jul 14 17:05:43.239378 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:43.239345    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t8zcm"]
Jul 14 17:05:43.239574 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:43.239456    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:43.239574 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:43.239560    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:43.243534 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:43.243512    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-plxbp"]
Jul 14 17:05:43.243673 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:43.243599    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:43.243729 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:43.243684    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:43.244354 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:43.244335    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zg6rn"]
Jul 14 17:05:43.244417 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:43.244407    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:43.244497 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:43.244482    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:44.164627 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:44.164538    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ea08ac1-2015-4bf9-b169-8b93b02530d3" containerID="0e31c7452865b491a7825298ecc892bea498afaa0f9bc17ea71b07c414bd9148" exitCode=0
Jul 14 17:05:44.165290 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:44.164632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ng8k2" event={"ID":"9ea08ac1-2015-4bf9-b169-8b93b02530d3","Type":"ContainerDied","Data":"0e31c7452865b491a7825298ecc892bea498afaa0f9bc17ea71b07c414bd9148"}
Jul 14 17:05:44.875765 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:44.875728    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:44.875954 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:44.875729    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:44.875954 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:44.875887    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:44.875954 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:44.875728    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:44.875954 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:44.875925    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:44.876175 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:44.876063    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:46.285906 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:46.285876    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8wp2t_f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e/dns-node-resolver/0.log"
Jul 14 17:05:46.876153 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:46.875941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:46.876305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:46.875941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:46.876305 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:46.876256    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plxbp" podUID="7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8"
Jul 14 17:05:46.876414 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:46.876360    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zg6rn" podUID="b40f2043-fe66-46e3-8d61-2567218a4c1e"
Jul 14 17:05:46.876414 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:46.875944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:46.876507 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:46.876463    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t8zcm" podUID="ccda64ad-05c5-4aff-95f2-ea99d638c5aa"
Jul 14 17:05:47.414202 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:47.414108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:47.414202 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:47.414168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:47.414678 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:47.414280    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:47.414678 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:47.414294    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:47.414678 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:47.414362    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs podName:b40f2043-fe66-46e3-8d61-2567218a4c1e nodeName:}" failed. No retries permitted until 2026-07-14 17:06:19.41434017 +0000 UTC m=+66.161940029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs") pod "network-metrics-daemon-zg6rn" (UID: "b40f2043-fe66-46e3-8d61-2567218a4c1e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:05:47.414678 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:47.414381    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret podName:ccda64ad-05c5-4aff-95f2-ea99d638c5aa nodeName:}" failed. No retries permitted until 2026-07-14 17:06:19.414372695 +0000 UTC m=+66.161972552 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret") pod "global-pull-secret-syncer-t8zcm" (UID: "ccda64ad-05c5-4aff-95f2-ea99d638c5aa") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:05:47.460061 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:47.460032    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ksph_4cefdd01-3531-4e69-89da-a20d63e8f853/node-ca/0.log"
Jul 14 17:05:47.515494 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:47.515452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlt67\" (UniqueName: \"kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67\") pod \"network-check-target-plxbp\" (UID: \"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8\") " pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:47.515722 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:47.515573    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:05:47.515722 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:47.515635    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:05:47.515722 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:47.515648    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rlt67 for pod openshift-network-diagnostics/network-check-target-plxbp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:47.515722 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:47.515711    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67 podName:7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:19.515692613 +0000 UTC m=+66.263292474 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rlt67" (UniqueName: "kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67") pod "network-check-target-plxbp" (UID: "7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:05:48.556960 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.556939    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-134.ec2.internal" event="NodeReady"
Jul 14 17:05:48.557306 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.557048    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 14 17:05:48.602937 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.602903    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-92pf7"]
Jul 14 17:05:48.623739 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.623706    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jc8t9"]
Jul 14 17:05:48.623900 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.623801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.627808 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.627785    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8v8vb\""
Jul 14 17:05:48.628109 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.628092    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 14 17:05:48.628356 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.628343    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 14 17:05:48.639938 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.639912    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-92pf7"]
Jul 14 17:05:48.640047 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.639959    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jc8t9"]
Jul 14 17:05:48.640047 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.639991    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-txz5w"]
Jul 14 17:05:48.640047 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.640031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.643305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.643245    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 14 17:05:48.643305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.643254    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 14 17:05:48.643305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.643296    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 14 17:05:48.643634 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.643349    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 14 17:05:48.643634 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.643255    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6x48r\""
Jul 14 17:05:48.659891 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.659862    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-txz5w"]
Jul 14 17:05:48.659891 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.659876    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-txz5w"
Jul 14 17:05:48.662360 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.662340    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 14 17:05:48.662469 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.662455    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 14 17:05:48.662615 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.662595    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vhkzd\""
Jul 14 17:05:48.662754 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.662670    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 14 17:05:48.722264 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.722230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7sd5\" (UniqueName: \"kubernetes.io/projected/fa711789-8cf8-4e24-86ca-da997986b21b-kube-api-access-h7sd5\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.722381 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.722272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.722381 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.722300    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.722381 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.722357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b13c40c-b85c-471d-9488-45f98f89c072-crio-socket\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.722381 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.722380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.722518 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.722402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa711789-8cf8-4e24-86ca-da997986b21b-tmp-dir\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.722518 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.722432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa711789-8cf8-4e24-86ca-da997986b21b-config-volume\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.722518 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.722452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx4sk\" (UniqueName: \"kubernetes.io/projected/4b13c40c-b85c-471d-9488-45f98f89c072-kube-api-access-rx4sk\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.722518 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.722496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b13c40c-b85c-471d-9488-45f98f89c072-data-volume\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.823015 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.822927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7sd5\" (UniqueName: \"kubernetes.io/projected/fa711789-8cf8-4e24-86ca-da997986b21b-kube-api-access-h7sd5\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.823015 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.822968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.823015 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.822990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823019    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94-cert\") pod \"ingress-canary-txz5w\" (UID: \"cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94\") " pod="openshift-ingress-canary/ingress-canary-txz5w"
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:48.823067    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b13c40c-b85c-471d-9488-45f98f89c072-crio-socket\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:48.823092    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa711789-8cf8-4e24-86ca-da997986b21b-tmp-dir\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:48.823146    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm podName:4b13c40c-b85c-471d-9488-45f98f89c072 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:49.323125286 +0000 UTC m=+36.070725158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jc8t9" (UID: "4b13c40c-b85c-471d-9488-45f98f89c072") : configmap "kube-rbac-proxy" not found
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:48.823172    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls podName:4b13c40c-b85c-471d-9488-45f98f89c072 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:49.323161467 +0000 UTC m=+36.070761323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jc8t9" (UID: "4b13c40c-b85c-471d-9488-45f98f89c072") : secret "insights-runtime-extractor-tls" not found
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa711789-8cf8-4e24-86ca-da997986b21b-config-volume\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:48.823233    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:05:48.823274 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh8q2\" (UniqueName: \"kubernetes.io/projected/cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94-kube-api-access-qh8q2\") pod \"ingress-canary-txz5w\" (UID: \"cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94\") " pod="openshift-ingress-canary/ingress-canary-txz5w"
Jul 14 17:05:48.823759 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b13c40c-b85c-471d-9488-45f98f89c072-crio-socket\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.823759 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx4sk\" (UniqueName: \"kubernetes.io/projected/4b13c40c-b85c-471d-9488-45f98f89c072-kube-api-access-rx4sk\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.823759 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:48.823347    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls podName:fa711789-8cf8-4e24-86ca-da997986b21b nodeName:}" failed. No retries permitted until 2026-07-14 17:05:49.323327634 +0000 UTC m=+36.070927502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls") pod "dns-default-92pf7" (UID: "fa711789-8cf8-4e24-86ca-da997986b21b") : secret "dns-default-metrics-tls" not found
Jul 14 17:05:48.823759 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b13c40c-b85c-471d-9488-45f98f89c072-data-volume\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.823759 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823461    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa711789-8cf8-4e24-86ca-da997986b21b-tmp-dir\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.823759 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b13c40c-b85c-471d-9488-45f98f89c072-data-volume\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.823759 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.823726    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa711789-8cf8-4e24-86ca-da997986b21b-config-volume\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.838340 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.838318    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7sd5\" (UniqueName: \"kubernetes.io/projected/fa711789-8cf8-4e24-86ca-da997986b21b-kube-api-access-h7sd5\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:48.838458 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.838376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx4sk\" (UniqueName: \"kubernetes.io/projected/4b13c40c-b85c-471d-9488-45f98f89c072-kube-api-access-rx4sk\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:48.878970 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.876320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:05:48.878970 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.876328    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:05:48.878970 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.876338    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:05:48.879777 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.879756    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 17:05:48.879932 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.879906    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 17:05:48.880054 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.879912    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7wgm\""
Jul 14 17:05:48.880176 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.880151    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 17:05:48.880243 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.880161    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tv9jb\""
Jul 14 17:05:48.880295 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.880275    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 17:05:48.923939 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.923902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94-cert\") pod \"ingress-canary-txz5w\" (UID: \"cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94\") " pod="openshift-ingress-canary/ingress-canary-txz5w"
Jul 14 17:05:48.924113 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.923994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh8q2\" (UniqueName: \"kubernetes.io/projected/cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94-kube-api-access-qh8q2\") pod \"ingress-canary-txz5w\" (UID: \"cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94\") " pod="openshift-ingress-canary/ingress-canary-txz5w"
Jul 14 17:05:48.926448 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.926425    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94-cert\") pod \"ingress-canary-txz5w\" (UID: \"cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94\") " pod="openshift-ingress-canary/ingress-canary-txz5w"
Jul 14 17:05:48.933190 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.933168    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh8q2\" (UniqueName: \"kubernetes.io/projected/cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94-kube-api-access-qh8q2\") pod \"ingress-canary-txz5w\" (UID: \"cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94\") " pod="openshift-ingress-canary/ingress-canary-txz5w"
Jul 14 17:05:48.968412 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:48.968385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-txz5w"
Jul 14 17:05:49.153775 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:49.153743    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-txz5w"]
Jul 14 17:05:49.163536 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:49.163505    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd76e29d_8191_4fba_ab9f_9d2ba2ec5f94.slice/crio-47b26b4305c6508f184867fbf907e57e554bc6ad43da7ebafd9e56308f407ec6 WatchSource:0}: Error finding container 47b26b4305c6508f184867fbf907e57e554bc6ad43da7ebafd9e56308f407ec6: Status 404 returned error can't find the container with id 47b26b4305c6508f184867fbf907e57e554bc6ad43da7ebafd9e56308f407ec6
Jul 14 17:05:49.178429 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:49.178398    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ea08ac1-2015-4bf9-b169-8b93b02530d3" containerID="5a2ecbf309482b5e5de23ec3ea154d7e6cd83714bfb0052ce75fe275823f00aa" exitCode=0
Jul 14 17:05:49.178569 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:49.178472    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ng8k2" event={"ID":"9ea08ac1-2015-4bf9-b169-8b93b02530d3","Type":"ContainerDied","Data":"5a2ecbf309482b5e5de23ec3ea154d7e6cd83714bfb0052ce75fe275823f00aa"}
Jul 14 17:05:49.179513 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:49.179490    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-txz5w" event={"ID":"cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94","Type":"ContainerStarted","Data":"47b26b4305c6508f184867fbf907e57e554bc6ad43da7ebafd9e56308f407ec6"}
Jul 14 17:05:49.328699 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:49.328625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:49.328699 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:49.328660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:49.328925 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:49.328743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:49.328925 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:49.328766    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 17:05:49.328925 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:49.328829    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm podName:4b13c40c-b85c-471d-9488-45f98f89c072 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:50.328810664 +0000 UTC m=+37.076410519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jc8t9" (UID: "4b13c40c-b85c-471d-9488-45f98f89c072") : configmap "kube-rbac-proxy" not found
Jul 14 17:05:49.329084 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:49.328935    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 17:05:49.329084 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:49.328979    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:05:49.329084 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:49.328993    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls podName:4b13c40c-b85c-471d-9488-45f98f89c072 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:50.328976228 +0000 UTC m=+37.076576083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jc8t9" (UID: "4b13c40c-b85c-471d-9488-45f98f89c072") : secret "insights-runtime-extractor-tls" not found
Jul 14 17:05:49.329084 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:49.329025    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls podName:fa711789-8cf8-4e24-86ca-da997986b21b nodeName:}" failed. No retries permitted until 2026-07-14 17:05:50.329009894 +0000 UTC m=+37.076609777 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls") pod "dns-default-92pf7" (UID: "fa711789-8cf8-4e24-86ca-da997986b21b") : secret "dns-default-metrics-tls" not found
Jul 14 17:05:50.184944 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:50.184910    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ea08ac1-2015-4bf9-b169-8b93b02530d3" containerID="18c126f4b5e01f6d824fcd316abaa6efbb2dc4e2e0200731a0a0dfebd631da87" exitCode=0
Jul 14 17:05:50.185334 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:50.184974    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ng8k2" event={"ID":"9ea08ac1-2015-4bf9-b169-8b93b02530d3","Type":"ContainerDied","Data":"18c126f4b5e01f6d824fcd316abaa6efbb2dc4e2e0200731a0a0dfebd631da87"}
Jul 14 17:05:50.336646 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:50.336608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:50.336646 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:50.336647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:50.336915 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:50.336701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:50.336915 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:50.336704    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 17:05:50.336915 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:50.336768    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm podName:4b13c40c-b85c-471d-9488-45f98f89c072 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:52.336746689 +0000 UTC m=+39.084346564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jc8t9" (UID: "4b13c40c-b85c-471d-9488-45f98f89c072") : configmap "kube-rbac-proxy" not found
Jul 14 17:05:50.336915 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:50.336827    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:05:50.336915 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:50.336828    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 17:05:50.336915 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:50.336905    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls podName:4b13c40c-b85c-471d-9488-45f98f89c072 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:52.336887075 +0000 UTC m=+39.084486934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jc8t9" (UID: "4b13c40c-b85c-471d-9488-45f98f89c072") : secret "insights-runtime-extractor-tls" not found
Jul 14 17:05:50.337135 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:50.336925    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls podName:fa711789-8cf8-4e24-86ca-da997986b21b nodeName:}" failed. No retries permitted until 2026-07-14 17:05:52.336916256 +0000 UTC m=+39.084516119 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls") pod "dns-default-92pf7" (UID: "fa711789-8cf8-4e24-86ca-da997986b21b") : secret "dns-default-metrics-tls" not found
Jul 14 17:05:51.191696 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:51.191664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ng8k2" event={"ID":"9ea08ac1-2015-4bf9-b169-8b93b02530d3","Type":"ContainerStarted","Data":"81814cd57c6afbfc1867cedd81511584a4d097bf134ce217671373a5294c5adc"}
Jul 14 17:05:51.219789 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:51.219720    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ng8k2" podStartSLOduration=9.281797892 podStartE2EDuration="38.219698887s" totalImagesPullingTime="28.937900995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:51.21829735 +0000 UTC m=+37.965897258" watchObservedRunningTime="2026-07-14 17:05:51.219698887 +0000 UTC m=+37.967298765"
Jul 14 17:05:52.352700 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:52.352647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:52.352700 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:52.352698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:52.353428 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:52.352753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:52.353428 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:52.352952    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:05:52.353428 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:52.353000    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls podName:fa711789-8cf8-4e24-86ca-da997986b21b nodeName:}" failed. No retries permitted until 2026-07-14 17:05:56.352987039 +0000 UTC m=+43.100586894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls") pod "dns-default-92pf7" (UID: "fa711789-8cf8-4e24-86ca-da997986b21b") : secret "dns-default-metrics-tls" not found
Jul 14 17:05:52.353428 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:52.353084    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 17:05:52.353428 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:52.353141    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls podName:4b13c40c-b85c-471d-9488-45f98f89c072 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:56.353124611 +0000 UTC m=+43.100724466 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jc8t9" (UID: "4b13c40c-b85c-471d-9488-45f98f89c072") : secret "insights-runtime-extractor-tls" not found
Jul 14 17:05:52.353428 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:52.353288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b13c40c-b85c-471d-9488-45f98f89c072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:54.200890 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:54.200838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-txz5w" event={"ID":"cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94","Type":"ContainerStarted","Data":"8d336a2547475a0c8c9f62bcb279b3630ec4d0e709911a711fdeb0b8c76337e1"}
Jul 14 17:05:56.384162 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:56.384115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:56.384636 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:56.384229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:05:56.384636 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:56.384339    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 17:05:56.384636 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:05:56.384410    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls podName:4b13c40c-b85c-471d-9488-45f98f89c072 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:04.384390134 +0000 UTC m=+51.131989996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jc8t9" (UID: "4b13c40c-b85c-471d-9488-45f98f89c072") : secret "insights-runtime-extractor-tls" not found
Jul 14 17:05:56.400500 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:56.400468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa711789-8cf8-4e24-86ca-da997986b21b-metrics-tls\") pod \"dns-default-92pf7\" (UID: \"fa711789-8cf8-4e24-86ca-da997986b21b\") " pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:56.436187 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:56.436138    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:56.594123 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:56.594071    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-txz5w" podStartSLOduration=4.161901847 podStartE2EDuration="8.594055166s" totalImagesPullingTime="4.432153319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:54.260965672 +0000 UTC m=+41.008565548" watchObservedRunningTime="2026-07-14 17:05:56.594055166 +0000 UTC m=+43.341655043"
Jul 14 17:05:56.594347 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:56.594285    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-92pf7"]
Jul 14 17:05:56.596858 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:05:56.596813    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa711789_8cf8_4e24_86ca_da997986b21b.slice/crio-26ca11fd3578c890a3c5de7f0c1b6f42591c9333aa3b49f04e808d0e8109433e WatchSource:0}: Error finding container 26ca11fd3578c890a3c5de7f0c1b6f42591c9333aa3b49f04e808d0e8109433e: Status 404 returned error can't find the container with id 26ca11fd3578c890a3c5de7f0c1b6f42591c9333aa3b49f04e808d0e8109433e
Jul 14 17:05:57.209973 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:57.209937    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-92pf7" event={"ID":"fa711789-8cf8-4e24-86ca-da997986b21b","Type":"ContainerStarted","Data":"26ca11fd3578c890a3c5de7f0c1b6f42591c9333aa3b49f04e808d0e8109433e"}
Jul 14 17:05:59.216996 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:59.216963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-92pf7" event={"ID":"fa711789-8cf8-4e24-86ca-da997986b21b","Type":"ContainerStarted","Data":"f5975d52ac6bd2366f1bf4282d19acaff6c098c680539bef71d5c626dcb7a5eb"}
Jul 14 17:05:59.217381 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:59.217019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-92pf7" event={"ID":"fa711789-8cf8-4e24-86ca-da997986b21b","Type":"ContainerStarted","Data":"195bc821fff8efd9c277eb6199e51591964c4925e2cdf1239926e54fe98ce803"}
Jul 14 17:05:59.217381 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:59.217057    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-92pf7"
Jul 14 17:05:59.238409 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:05:59.238357    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-92pf7" podStartSLOduration=8.908731687 podStartE2EDuration="11.238340078s" totalImagesPullingTime="2.329608391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:59.237292541 +0000 UTC m=+45.984892448" watchObservedRunningTime="2026-07-14 17:05:59.238340078 +0000 UTC m=+45.985939933"
Jul 14 17:06:04.441547 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:04.441501    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:06:04.451933 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:04.451901    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b13c40c-b85c-471d-9488-45f98f89c072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jc8t9\" (UID: \"4b13c40c-b85c-471d-9488-45f98f89c072\") " pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:06:04.550611 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:04.550574    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jc8t9"
Jul 14 17:06:04.685084 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:04.685049    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jc8t9"]
Jul 14 17:06:04.688118 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:06:04.688093    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b13c40c_b85c_471d_9488_45f98f89c072.slice/crio-e2fb7fcb48f9d4dd810f6119ce9bc3b66ad24dc0118c684b762dd1a10000a3b4 WatchSource:0}: Error finding container e2fb7fcb48f9d4dd810f6119ce9bc3b66ad24dc0118c684b762dd1a10000a3b4: Status 404 returned error can't find the container with id e2fb7fcb48f9d4dd810f6119ce9bc3b66ad24dc0118c684b762dd1a10000a3b4
Jul 14 17:06:05.233054 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:05.233016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jc8t9" event={"ID":"4b13c40c-b85c-471d-9488-45f98f89c072","Type":"ContainerStarted","Data":"e2fb7fcb48f9d4dd810f6119ce9bc3b66ad24dc0118c684b762dd1a10000a3b4"}
Jul 14 17:06:07.247255 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.247222    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"]
Jul 14 17:06:07.256638 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.256607    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"]
Jul 14 17:06:07.256794 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.256731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"
Jul 14 17:06:07.258787 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.258764    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.259986 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.259967    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 14 17:06:07.260776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.260755    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 14 17:06:07.261030 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.260819    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 14 17:06:07.261249 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.261218    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 14 17:06:07.261503 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.261485    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 14 17:06:07.261605 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.261542    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 14 17:06:07.262113 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.261705    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 14 17:06:07.262113 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.261969    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jdbf9\""
Jul 14 17:06:07.262113 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.262005    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 14 17:06:07.262113 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.262022    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"]
Jul 14 17:06:07.282661 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.282636    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"]
Jul 14 17:06:07.365861 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.365807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-hub\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.366036 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.365885    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.366036 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.365948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7ebd21-ad67-477d-8396-ca03433814fc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.366036 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.365985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2rk4\" (UniqueName: \"kubernetes.io/projected/b8557644-d4d7-49bb-a3a5-2d2f8b81472d-kube-api-access-k2rk4\") pod \"managed-serviceaccount-addon-agent-74684f5778-lvdpm\" (UID: \"b8557644-d4d7-49bb-a3a5-2d2f8b81472d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"
Jul 14 17:06:07.366184 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.366050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8mlq\" (UniqueName: \"kubernetes.io/projected/bd7ebd21-ad67-477d-8396-ca03433814fc-kube-api-access-z8mlq\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.366184 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.366095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8557644-d4d7-49bb-a3a5-2d2f8b81472d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74684f5778-lvdpm\" (UID: \"b8557644-d4d7-49bb-a3a5-2d2f8b81472d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"
Jul 14 17:06:07.366184 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.366121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.366184 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.366150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-ca\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.467265 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.467225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-hub\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.467265 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.467266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.467493 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.467314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7ebd21-ad67-477d-8396-ca03433814fc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.467493 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.467348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2rk4\" (UniqueName: \"kubernetes.io/projected/b8557644-d4d7-49bb-a3a5-2d2f8b81472d-kube-api-access-k2rk4\") pod \"managed-serviceaccount-addon-agent-74684f5778-lvdpm\" (UID: \"b8557644-d4d7-49bb-a3a5-2d2f8b81472d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"
Jul 14 17:06:07.467493 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.467383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8mlq\" (UniqueName: \"kubernetes.io/projected/bd7ebd21-ad67-477d-8396-ca03433814fc-kube-api-access-z8mlq\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.467493 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.467417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8557644-d4d7-49bb-a3a5-2d2f8b81472d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74684f5778-lvdpm\" (UID: \"b8557644-d4d7-49bb-a3a5-2d2f8b81472d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"
Jul 14 17:06:07.467493 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.467438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.467493 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.467468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-ca\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.468751 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.468716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7ebd21-ad67-477d-8396-ca03433814fc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.470418 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.470390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.470560 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.470503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-hub\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.470746 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.470704    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8557644-d4d7-49bb-a3a5-2d2f8b81472d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74684f5778-lvdpm\" (UID: \"b8557644-d4d7-49bb-a3a5-2d2f8b81472d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"
Jul 14 17:06:07.470979 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.470957    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-ca\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.471102 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.471086    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd7ebd21-ad67-477d-8396-ca03433814fc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.483864 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.483820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8mlq\" (UniqueName: \"kubernetes.io/projected/bd7ebd21-ad67-477d-8396-ca03433814fc-kube-api-access-z8mlq\") pod \"cluster-proxy-proxy-agent-6b98c4db9b-fpkhf\" (UID: \"bd7ebd21-ad67-477d-8396-ca03433814fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:07.484365 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.484346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2rk4\" (UniqueName: \"kubernetes.io/projected/b8557644-d4d7-49bb-a3a5-2d2f8b81472d-kube-api-access-k2rk4\") pod \"managed-serviceaccount-addon-agent-74684f5778-lvdpm\" (UID: \"b8557644-d4d7-49bb-a3a5-2d2f8b81472d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"
Jul 14 17:06:07.577512 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.577421    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"
Jul 14 17:06:07.584164 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:07.584140    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"
Jul 14 17:06:09.221462 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:09.221429    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-92pf7"
Jul 14 17:06:09.246284 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:09.246224    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jc8t9" event={"ID":"4b13c40c-b85c-471d-9488-45f98f89c072","Type":"ContainerStarted","Data":"ca8695b09f1b6738b4dbd23c4fbc00f7cb43cefb7aa12c6faf072704e84bede3"}
Jul 14 17:06:09.248147 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:09.248122    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf"]
Jul 14 17:06:09.250698 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:06:09.250661    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd7ebd21_ad67_477d_8396_ca03433814fc.slice/crio-2f946cb7217f4f6d287a928353d94c25486559e0cab45d8cc7db16c8b53c8872 WatchSource:0}: Error finding container 2f946cb7217f4f6d287a928353d94c25486559e0cab45d8cc7db16c8b53c8872: Status 404 returned error can't find the container with id 2f946cb7217f4f6d287a928353d94c25486559e0cab45d8cc7db16c8b53c8872
Jul 14 17:06:09.261715 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:09.261167    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm"]
Jul 14 17:06:09.266538 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:06:09.266129    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8557644_d4d7_49bb_a3a5_2d2f8b81472d.slice/crio-1c9afdfe91549536129c178945a39d0e2bac236175f20f3b558726a6573bda8e WatchSource:0}: Error finding container 1c9afdfe91549536129c178945a39d0e2bac236175f20f3b558726a6573bda8e: Status 404 returned error can't find the container with id 1c9afdfe91549536129c178945a39d0e2bac236175f20f3b558726a6573bda8e
Jul 14 17:06:10.251607 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:10.251571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm" event={"ID":"b8557644-d4d7-49bb-a3a5-2d2f8b81472d","Type":"ContainerStarted","Data":"1c9afdfe91549536129c178945a39d0e2bac236175f20f3b558726a6573bda8e"}
Jul 14 17:06:10.253357 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:10.253329    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf" event={"ID":"bd7ebd21-ad67-477d-8396-ca03433814fc","Type":"ContainerStarted","Data":"2f946cb7217f4f6d287a928353d94c25486559e0cab45d8cc7db16c8b53c8872"}
Jul 14 17:06:11.260026 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:11.259986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jc8t9" event={"ID":"4b13c40c-b85c-471d-9488-45f98f89c072","Type":"ContainerStarted","Data":"21a3ac4565e446e08b56cef21b6aabbbcea114809f24145eda9d8279c10d7590"}
Jul 14 17:06:14.177311 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:14.177280    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59wt"
Jul 14 17:06:14.273202 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:14.273151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf" event={"ID":"bd7ebd21-ad67-477d-8396-ca03433814fc","Type":"ContainerStarted","Data":"94d9c6110a57fab039ecea6bd8f142c8488d84fd0acb0f731ee9a1ac1e796ead"}
Jul 14 17:06:14.273357 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:14.273212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf" event={"ID":"bd7ebd21-ad67-477d-8396-ca03433814fc","Type":"ContainerStarted","Data":"f8c8642c4d3169f18add4c41f56e10d044ceab2fe84fbd853a3ad7f3b634ba3e"}
Jul 14 17:06:14.275317 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:14.275273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm" event={"ID":"b8557644-d4d7-49bb-a3a5-2d2f8b81472d","Type":"ContainerStarted","Data":"eda381256ecd589db5b13f27d880ae9df305f2a2aa6124c8caaa2bfa773ddc39"}
Jul 14 17:06:14.293951 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:14.293898    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74684f5778-lvdpm" podStartSLOduration=3.096698765 podStartE2EDuration="7.293880749s" totalImagesPullingTime="4.197181984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:14.293066839 +0000 UTC m=+61.040666717" watchObservedRunningTime="2026-07-14 17:06:14.293880749 +0000 UTC m=+61.041480623"
Jul 14 17:06:15.280593 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:15.280557    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf" event={"ID":"bd7ebd21-ad67-477d-8396-ca03433814fc","Type":"ContainerStarted","Data":"6b315c99e7028177a971293c41d43c5fc69f5482fcf837fadf31530c9de62d32"}
Jul 14 17:06:15.282597 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:15.282568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jc8t9" event={"ID":"4b13c40c-b85c-471d-9488-45f98f89c072","Type":"ContainerStarted","Data":"49ed007eedec5c28285ef6bcfacb73f03c9c45b927947bcac1ed57c395c7a8ca"}
Jul 14 17:06:15.307493 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:15.307443    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b98c4db9b-fpkhf" podStartSLOduration=4.079602578 podStartE2EDuration="8.307426843s" totalImagesPullingTime="4.227824265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:15.306037803 +0000 UTC m=+62.053637680" watchObservedRunningTime="2026-07-14 17:06:15.307426843 +0000 UTC m=+62.055026769"
Jul 14 17:06:15.335386 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:15.335334    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jc8t9" podStartSLOduration=17.467395918 podStartE2EDuration="27.335316236s" totalImagesPullingTime="9.867920318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:15.333215394 +0000 UTC m=+62.080815283" watchObservedRunningTime="2026-07-14 17:06:15.335316236 +0000 UTC m=+62.082916116"
Jul 14 17:06:19.466950 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.466909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:06:19.467322 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.466962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:06:19.469608 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.469583    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 17:06:19.469700 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.469583    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 17:06:19.480426 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.480389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40f2043-fe66-46e3-8d61-2567218a4c1e-metrics-certs\") pod \"network-metrics-daemon-zg6rn\" (UID: \"b40f2043-fe66-46e3-8d61-2567218a4c1e\") " pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:06:19.480426 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.480426    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccda64ad-05c5-4aff-95f2-ea99d638c5aa-original-pull-secret\") pod \"global-pull-secret-syncer-t8zcm\" (UID: \"ccda64ad-05c5-4aff-95f2-ea99d638c5aa\") " pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:06:19.490771 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.490749    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7wgm\""
Jul 14 17:06:19.494040 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.494025    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8zcm"
Jul 14 17:06:19.499218 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.499199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zg6rn"
Jul 14 17:06:19.515401 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.515371    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-2cqrq"]
Jul 14 17:06:19.531114 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.531087    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-2cqrq"]
Jul 14 17:06:19.531241 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.531187    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.534292 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.534270    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 14 17:06:19.534414 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.534324    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 14 17:06:19.534482 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.534413    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nq2lr\""
Jul 14 17:06:19.534535 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.534279    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 14 17:06:19.534727 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.534711    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 14 17:06:19.535228 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.535203    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 14 17:06:19.568110 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.568078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlt67\" (UniqueName: \"kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67\") pod \"network-check-target-plxbp\" (UID: \"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8\") " pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:06:19.568247 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.568120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.568247 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.568151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.568247 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.568178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-metrics-client-ca\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.568247 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.568217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqrtc\" (UniqueName: \"kubernetes.io/projected/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-kube-api-access-qqrtc\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.570964 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.570942    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 17:06:19.582431 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.582199    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 17:06:19.593541 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.593476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlt67\" (UniqueName: \"kubernetes.io/projected/7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8-kube-api-access-rlt67\") pod \"network-check-target-plxbp\" (UID: \"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8\") " pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:06:19.669509 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.669467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.669748 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.669519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.669748 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.669561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-metrics-client-ca\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.669748 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.669607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrtc\" (UniqueName: \"kubernetes.io/projected/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-kube-api-access-qqrtc\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.669966 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:06:19.669952    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 14 17:06:19.670053 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:06:19.670028    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-tls podName:6ca7ace3-a7f3-4381-a322-ce22907f0bd7 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:20.170004664 +0000 UTC m=+66.917604519 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-tls") pod "prometheus-operator-64785b6675-2cqrq" (UID: "6ca7ace3-a7f3-4381-a322-ce22907f0bd7") : secret "prometheus-operator-tls" not found
Jul 14 17:06:19.670446 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.670414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-metrics-client-ca\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.670818 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.670799    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t8zcm"]
Jul 14 17:06:19.672628 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.672604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.674228 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:06:19.674205    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccda64ad_05c5_4aff_95f2_ea99d638c5aa.slice/crio-63c4a39911e8de650f536564bcc1d40e2d065b5d29387da10f285930e412b1b1 WatchSource:0}: Error finding container 63c4a39911e8de650f536564bcc1d40e2d065b5d29387da10f285930e412b1b1: Status 404 returned error can't find the container with id 63c4a39911e8de650f536564bcc1d40e2d065b5d29387da10f285930e412b1b1
Jul 14 17:06:19.678374 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.678350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqrtc\" (UniqueName: \"kubernetes.io/projected/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-kube-api-access-qqrtc\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:19.686784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.686760    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zg6rn"]
Jul 14 17:06:19.689829 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:06:19.689812    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40f2043_fe66_46e3_8d61_2567218a4c1e.slice/crio-55d66a03f1105f1feca6dc811f624e2bb266364daa39489418c5c4635e03228a WatchSource:0}: Error finding container 55d66a03f1105f1feca6dc811f624e2bb266364daa39489418c5c4635e03228a: Status 404 returned error can't find the container with id 55d66a03f1105f1feca6dc811f624e2bb266364daa39489418c5c4635e03228a
Jul 14 17:06:19.800908 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.800808    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tv9jb\""
Jul 14 17:06:19.809258 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.809238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:06:19.962506 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:19.962357    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-plxbp"]
Jul 14 17:06:19.965130 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:06:19.965096    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d07a9b0_d7ae_43c6_8fa1_fc2e80aefbe8.slice/crio-cb948965c8567fe451f97426280b28f6a7c89ee4949aa63c6e5922c2c9b7e969 WatchSource:0}: Error finding container cb948965c8567fe451f97426280b28f6a7c89ee4949aa63c6e5922c2c9b7e969: Status 404 returned error can't find the container with id cb948965c8567fe451f97426280b28f6a7c89ee4949aa63c6e5922c2c9b7e969
Jul 14 17:06:20.173560 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:20.173479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:20.173755 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:06:20.173680    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 14 17:06:20.173822 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:06:20.173762    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-tls podName:6ca7ace3-a7f3-4381-a322-ce22907f0bd7 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:21.173742513 +0000 UTC m=+67.921342380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-tls") pod "prometheus-operator-64785b6675-2cqrq" (UID: "6ca7ace3-a7f3-4381-a322-ce22907f0bd7") : secret "prometheus-operator-tls" not found
Jul 14 17:06:20.298507 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:20.298461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zg6rn" event={"ID":"b40f2043-fe66-46e3-8d61-2567218a4c1e","Type":"ContainerStarted","Data":"55d66a03f1105f1feca6dc811f624e2bb266364daa39489418c5c4635e03228a"}
Jul 14 17:06:20.300133 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:20.299993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-plxbp" event={"ID":"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8","Type":"ContainerStarted","Data":"cb948965c8567fe451f97426280b28f6a7c89ee4949aa63c6e5922c2c9b7e969"}
Jul 14 17:06:20.301596 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:20.301538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t8zcm" event={"ID":"ccda64ad-05c5-4aff-95f2-ea99d638c5aa","Type":"ContainerStarted","Data":"63c4a39911e8de650f536564bcc1d40e2d065b5d29387da10f285930e412b1b1"}
Jul 14 17:06:21.181001 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:21.180905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:21.184305 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:21.184277    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ace3-a7f3-4381-a322-ce22907f0bd7-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-2cqrq\" (UID: \"6ca7ace3-a7f3-4381-a322-ce22907f0bd7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:21.308568 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:21.308526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zg6rn" event={"ID":"b40f2043-fe66-46e3-8d61-2567218a4c1e","Type":"ContainerStarted","Data":"7e747a1a1b7905d38ccbe228f6672aaaf119d4a1571789bb79eeeb21620fd3fb"}
Jul 14 17:06:21.308750 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:21.308585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zg6rn" event={"ID":"b40f2043-fe66-46e3-8d61-2567218a4c1e","Type":"ContainerStarted","Data":"7d9d0936ea183634f46a9e8f872b1705de30bae61cd36bb8e582b4c07828c588"}
Jul 14 17:06:21.337226 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:21.337157    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zg6rn" podStartSLOduration=67.215900251 podStartE2EDuration="1m8.337133007s" totalImagesPullingTime="1.121232756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:21.33642806 +0000 UTC m=+68.084027938" watchObservedRunningTime="2026-07-14 17:06:21.337133007 +0000 UTC m=+68.084732880"
Jul 14 17:06:21.353562 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:21.353520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq"
Jul 14 17:06:21.516415 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:21.516379    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-2cqrq"]
Jul 14 17:06:21.522251 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:06:21.520835    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ca7ace3_a7f3_4381_a322_ce22907f0bd7.slice/crio-e7b53e2dba22e06b4d67e1a1ccfef2b2a0a50f8ab4120a256cfeb87947cccd6d WatchSource:0}: Error finding container e7b53e2dba22e06b4d67e1a1ccfef2b2a0a50f8ab4120a256cfeb87947cccd6d: Status 404 returned error can't find the container with id e7b53e2dba22e06b4d67e1a1ccfef2b2a0a50f8ab4120a256cfeb87947cccd6d
Jul 14 17:06:22.314224 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:22.314179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq" event={"ID":"6ca7ace3-a7f3-4381-a322-ce22907f0bd7","Type":"ContainerStarted","Data":"e7b53e2dba22e06b4d67e1a1ccfef2b2a0a50f8ab4120a256cfeb87947cccd6d"}
Jul 14 17:06:24.324149 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:24.324107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-plxbp" event={"ID":"7d07a9b0-d7ae-43c6-8fa1-fc2e80aefbe8","Type":"ContainerStarted","Data":"efddb16f1450e8fb067ee6af41c7e2c8836a0f8711c2e0e4642c07a11a178a39"}
Jul 14 17:06:24.324631 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:24.324313    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:06:24.326403 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:24.326368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq" event={"ID":"6ca7ace3-a7f3-4381-a322-ce22907f0bd7","Type":"ContainerStarted","Data":"4edb9e634a806912a190b57f39979e4d86e6b78d5504f65304ea2e6b9fef3e00"}
Jul 14 17:06:24.326544 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:24.326427    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq" event={"ID":"6ca7ace3-a7f3-4381-a322-ce22907f0bd7","Type":"ContainerStarted","Data":"107986e3cc6d2a956e16baae74f71856ee3a9e914f6a674c7d00e6f27fd99a63"}
Jul 14 17:06:24.344741 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:24.344688    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-plxbp" podStartSLOduration=67.810852967 podStartE2EDuration="1m11.344671584s" totalImagesPullingTime="3.533818617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:24.343645019 +0000 UTC m=+71.091244914" watchObservedRunningTime="2026-07-14 17:06:24.344671584 +0000 UTC m=+71.092271464"
Jul 14 17:06:24.365362 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:24.365302    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-2cqrq" podStartSLOduration=3.394603549 podStartE2EDuration="5.365279901s" totalImagesPullingTime="1.970676352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:24.36491066 +0000 UTC m=+71.112510557" watchObservedRunningTime="2026-07-14 17:06:24.365279901 +0000 UTC m=+71.112879779"
Jul 14 17:06:25.331385 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:25.331286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t8zcm" event={"ID":"ccda64ad-05c5-4aff-95f2-ea99d638c5aa","Type":"ContainerStarted","Data":"5eade7950b59083a69ccfd924d56d3bf9fb7567009b1d15dac60bc3a81d9e882"}
Jul 14 17:06:25.348971 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:25.348921    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t8zcm" podStartSLOduration=66.945574773 podStartE2EDuration="1m12.34890629s" totalImagesPullingTime="5.403331517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:25.348693818 +0000 UTC m=+72.096293710" watchObservedRunningTime="2026-07-14 17:06:25.34890629 +0000 UTC m=+72.096506145"
Jul 14 17:06:26.056467 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.056429    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m66gz"]
Jul 14 17:06:26.060574 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.060549    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.065343 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.065320    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 14 17:06:26.065479 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.065406    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pxj4z\""
Jul 14 17:06:26.065765 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.065748    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 14 17:06:26.070473 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.070455    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 14 17:06:26.120333 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.120298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-sys\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.120523 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.120342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-wtmp\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.120523 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.120364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.120523 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.120436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-metrics-client-ca\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.120523 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.120469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-tls\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.120523 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.120491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-accelerators-collector-config\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.120764 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.120582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-root\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.120764 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.120617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-textfile\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.120764 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.120645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkpqt\" (UniqueName: \"kubernetes.io/projected/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-kube-api-access-jkpqt\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221659 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-root\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221659 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-textfile\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkpqt\" (UniqueName: \"kubernetes.io/projected/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-kube-api-access-jkpqt\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-sys\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221750    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-wtmp\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-root\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-metrics-client-ca\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-sys\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-tls\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.221928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-accelerators-collector-config\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.222333 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.221956    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-wtmp\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.222333 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.222058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-textfile\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.222581 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.222551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-metrics-client-ca\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.222701 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.222596    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-accelerators-collector-config\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.224291 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.224268    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-tls\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.224488 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.224470    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.248508 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.248468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkpqt\" (UniqueName: \"kubernetes.io/projected/1a4a7edb-f4ed-4b31-9e02-e0e4fc116801-kube-api-access-jkpqt\") pod \"node-exporter-m66gz\" (UID: \"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801\") " pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.369676 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:26.369600    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m66gz"
Jul 14 17:06:26.384217 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:06:26.384190    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4a7edb_f4ed_4b31_9e02_e0e4fc116801.slice/crio-f8ba96934ad25c113d3419422c95a74261b3fb7f3aa4b4327ad3620d3d5810c8 WatchSource:0}: Error finding container f8ba96934ad25c113d3419422c95a74261b3fb7f3aa4b4327ad3620d3d5810c8: Status 404 returned error can't find the container with id f8ba96934ad25c113d3419422c95a74261b3fb7f3aa4b4327ad3620d3d5810c8
Jul 14 17:06:27.339836 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:27.339794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m66gz" event={"ID":"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801","Type":"ContainerStarted","Data":"f8ba96934ad25c113d3419422c95a74261b3fb7f3aa4b4327ad3620d3d5810c8"}
Jul 14 17:06:28.344551 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:28.344511    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a4a7edb-f4ed-4b31-9e02-e0e4fc116801" containerID="daaeff4a34ff2ebdec3971e7a7996afd6de38877b7d2ee6a0b6211365b8fb4be" exitCode=0
Jul 14 17:06:28.344551 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:28.344556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m66gz" event={"ID":"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801","Type":"ContainerDied","Data":"daaeff4a34ff2ebdec3971e7a7996afd6de38877b7d2ee6a0b6211365b8fb4be"}
Jul 14 17:06:29.351330 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:29.351284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m66gz" event={"ID":"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801","Type":"ContainerStarted","Data":"73013aa5244a617cb83dfa767814dc5fe9ab649f88748fdd814a4d7d971abba7"}
Jul 14 17:06:29.351717 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:29.351347    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m66gz" event={"ID":"1a4a7edb-f4ed-4b31-9e02-e0e4fc116801","Type":"ContainerStarted","Data":"7645c34fef432e26d277e71ea46799a8d2012f0585a0ab1eb44226f3c6a3ce53"}
Jul 14 17:06:30.444233 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.444170    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m66gz" podStartSLOduration=4.441502593 podStartE2EDuration="5.44415193s" totalImagesPullingTime="930.976115ms" totalInitContainerRuntime="71.673222ms" isStatefulPod=true podCreationTimestamp="2026-07-14 17:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:29.384228874 +0000 UTC m=+76.131828762" watchObservedRunningTime="2026-07-14 17:06:30.44415193 +0000 UTC m=+77.191751807"
Jul 14 17:06:30.444607 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.444570    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-58d965cb7-hzn25"]
Jul 14 17:06:30.447796 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.447780    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.450711 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.450688    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 14 17:06:30.450832 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.450740    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-q2fcg\""
Jul 14 17:06:30.451875 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.451835    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 14 17:06:30.452056 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.452008    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 14 17:06:30.452168 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.452149    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-frqmjfqpes7qm\""
Jul 14 17:06:30.452218 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.452190    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 14 17:06:30.457727 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.457702    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-58d965cb7-hzn25"]
Jul 14 17:06:30.555077 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.555041    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-secret-metrics-server-client-certs\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.555278 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.555091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-client-ca-bundle\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.555278 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.555158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-secret-metrics-server-tls\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.555278 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.555204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-metrics-server-audit-profiles\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.555278 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.555252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsz57\" (UniqueName: \"kubernetes.io/projected/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-kube-api-access-rsz57\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.555438 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.555279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.555438 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.555303    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-audit-log\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.656316 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.656282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-client-ca-bundle\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.656486 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.656329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-secret-metrics-server-tls\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.656486 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.656351    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-metrics-server-audit-profiles\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.656486 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.656385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsz57\" (UniqueName: \"kubernetes.io/projected/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-kube-api-access-rsz57\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.656486 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.656417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.656486 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.656441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-audit-log\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.656486 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.656482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-secret-metrics-server-client-certs\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.657016 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.656994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-audit-log\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.657479 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.657458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-metrics-server-audit-profiles\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.657551 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.657454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.659086 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.659058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-secret-metrics-server-tls\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.659202 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.659166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-secret-metrics-server-client-certs\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.659266 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.659199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-client-ca-bundle\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.665814 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.665791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsz57\" (UniqueName: \"kubernetes.io/projected/744a6ce0-c029-4ff7-a3e3-81e27b11d7c5-kube-api-access-rsz57\") pod \"metrics-server-58d965cb7-hzn25\" (UID: \"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5\") " pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.756763 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.756652    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:30.893226 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:30.893195    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-58d965cb7-hzn25"]
Jul 14 17:06:30.896136 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:06:30.896103    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod744a6ce0_c029_4ff7_a3e3_81e27b11d7c5.slice/crio-ad78cb1919eaf155c12f4c478722c034104858992ff6cf9c2d9f7e6114355791 WatchSource:0}: Error finding container ad78cb1919eaf155c12f4c478722c034104858992ff6cf9c2d9f7e6114355791: Status 404 returned error can't find the container with id ad78cb1919eaf155c12f4c478722c034104858992ff6cf9c2d9f7e6114355791
Jul 14 17:06:31.361173 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:31.361130    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-58d965cb7-hzn25" event={"ID":"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5","Type":"ContainerStarted","Data":"ad78cb1919eaf155c12f4c478722c034104858992ff6cf9c2d9f7e6114355791"}
Jul 14 17:06:33.371574 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:33.371532    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-58d965cb7-hzn25" event={"ID":"744a6ce0-c029-4ff7-a3e3-81e27b11d7c5","Type":"ContainerStarted","Data":"318af4dd5e9d80796364408c2e995d568a26dd71bf48ece687f40510228ad7ca"}
Jul 14 17:06:33.394949 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:33.394893    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-58d965cb7-hzn25" podStartSLOduration=1.841004973 podStartE2EDuration="3.39487602s" totalImagesPullingTime="1.553871047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:33.39431877 +0000 UTC m=+80.141918647" watchObservedRunningTime="2026-07-14 17:06:33.39487602 +0000 UTC m=+80.142475909"
Jul 14 17:06:50.757295 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:50.757256    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:50.757778 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:50.757323    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:06:55.334086 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:06:55.334054    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-plxbp"
Jul 14 17:07:10.763188 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:07:10.763153    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:07:10.767083 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:07:10.767060    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-58d965cb7-hzn25"
Jul 14 17:09:36.960952 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:36.960867    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"]
Jul 14 17:09:36.962958 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:36.962940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"
Jul 14 17:09:36.965613 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:36.965586    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 14 17:09:36.965613 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:36.965612    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 14 17:09:36.966892 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:36.966866    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-sg2nw\""
Jul 14 17:09:36.973585 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:36.973564    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"]
Jul 14 17:09:37.082651 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:37.082605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea15174e-431c-442a-a8be-252b32ccbb3f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-w7k9h\" (UID: \"ea15174e-431c-442a-a8be-252b32ccbb3f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"
Jul 14 17:09:37.082651 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:37.082656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfg52\" (UniqueName: \"kubernetes.io/projected/ea15174e-431c-442a-a8be-252b32ccbb3f-kube-api-access-bfg52\") pod \"cert-manager-cainjector-559cdd8c87-w7k9h\" (UID: \"ea15174e-431c-442a-a8be-252b32ccbb3f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"
Jul 14 17:09:37.183942 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:37.183906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea15174e-431c-442a-a8be-252b32ccbb3f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-w7k9h\" (UID: \"ea15174e-431c-442a-a8be-252b32ccbb3f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"
Jul 14 17:09:37.183942 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:37.183945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfg52\" (UniqueName: \"kubernetes.io/projected/ea15174e-431c-442a-a8be-252b32ccbb3f-kube-api-access-bfg52\") pod \"cert-manager-cainjector-559cdd8c87-w7k9h\" (UID: \"ea15174e-431c-442a-a8be-252b32ccbb3f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"
Jul 14 17:09:37.195736 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:37.195698    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea15174e-431c-442a-a8be-252b32ccbb3f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-w7k9h\" (UID: \"ea15174e-431c-442a-a8be-252b32ccbb3f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"
Jul 14 17:09:37.206595 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:37.206568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfg52\" (UniqueName: \"kubernetes.io/projected/ea15174e-431c-442a-a8be-252b32ccbb3f-kube-api-access-bfg52\") pod \"cert-manager-cainjector-559cdd8c87-w7k9h\" (UID: \"ea15174e-431c-442a-a8be-252b32ccbb3f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"
Jul 14 17:09:37.271907 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:37.271802    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"
Jul 14 17:09:37.404409 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:37.404366    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h"]
Jul 14 17:09:37.407353 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:09:37.407327    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea15174e_431c_442a_a8be_252b32ccbb3f.slice/crio-cdd715a9e67a911997dae9e13f0b0bcc7ad96f4f4b88904c718c07e5ef1be44b WatchSource:0}: Error finding container cdd715a9e67a911997dae9e13f0b0bcc7ad96f4f4b88904c718c07e5ef1be44b: Status 404 returned error can't find the container with id cdd715a9e67a911997dae9e13f0b0bcc7ad96f4f4b88904c718c07e5ef1be44b
Jul 14 17:09:37.974363 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:37.974319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h" event={"ID":"ea15174e-431c-442a-a8be-252b32ccbb3f","Type":"ContainerStarted","Data":"cdd715a9e67a911997dae9e13f0b0bcc7ad96f4f4b88904c718c07e5ef1be44b"}
Jul 14 17:09:41.992627 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:41.992585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h" event={"ID":"ea15174e-431c-442a-a8be-252b32ccbb3f","Type":"ContainerStarted","Data":"9d53e1b64f3e55ea232c78285516a228b7e052020e855f2cd0831dd78dd0bac7"}
Jul 14 17:09:42.015697 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:09:42.015643    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w7k9h" podStartSLOduration=2.482790407 podStartE2EDuration="6.015625946s" totalImagesPullingTime="3.532835539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:09:42.014609162 +0000 UTC m=+268.762209040" watchObservedRunningTime="2026-07-14 17:09:42.015625946 +0000 UTC m=+268.763225822"
Jul 14 17:10:07.960195 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:07.960161    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"]
Jul 14 17:10:07.962434 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:07.962413    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:07.965452 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:07.965434    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fv67k\""
Jul 14 17:10:07.965548 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:07.965482    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 14 17:10:07.965730 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:07.965706    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 14 17:10:07.965790 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:07.965736    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 14 17:10:07.965790 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:07.965784    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 14 17:10:07.983799 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:07.983767    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"]
Jul 14 17:10:08.009620 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.009591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88fee8f0-8236-4d6a-95aa-1355865b465a-tmp\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.009782 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.009647    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/88fee8f0-8236-4d6a-95aa-1355865b465a-manifests\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.009782 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.009706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/88fee8f0-8236-4d6a-95aa-1355865b465a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.009782 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.009774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mtp4\" (UniqueName: \"kubernetes.io/projected/88fee8f0-8236-4d6a-95aa-1355865b465a-kube-api-access-8mtp4\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.009931 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.009808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88fee8f0-8236-4d6a-95aa-1355865b465a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.110654 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.110618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mtp4\" (UniqueName: \"kubernetes.io/projected/88fee8f0-8236-4d6a-95aa-1355865b465a-kube-api-access-8mtp4\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.110654 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.110656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88fee8f0-8236-4d6a-95aa-1355865b465a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.110946 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.110682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88fee8f0-8236-4d6a-95aa-1355865b465a-tmp\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.110946 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.110719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/88fee8f0-8236-4d6a-95aa-1355865b465a-manifests\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.110946 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.110743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/88fee8f0-8236-4d6a-95aa-1355865b465a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.111225 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.111199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/88fee8f0-8236-4d6a-95aa-1355865b465a-manifests\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.113171 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.113146    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88fee8f0-8236-4d6a-95aa-1355865b465a-tmp\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.113336 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.113315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/88fee8f0-8236-4d6a-95aa-1355865b465a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.113377 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.113315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88fee8f0-8236-4d6a-95aa-1355865b465a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.123179 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.123156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mtp4\" (UniqueName: \"kubernetes.io/projected/88fee8f0-8236-4d6a-95aa-1355865b465a-kube-api-access-8mtp4\") pod \"opendatahub-operator-controller-manager-7655f47d9d-k7fk7\" (UID: \"88fee8f0-8236-4d6a-95aa-1355865b465a\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.272198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.272096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:08.418378 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:08.418345    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"]
Jul 14 17:10:08.421766 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:10:08.421733    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88fee8f0_8236_4d6a_95aa_1355865b465a.slice/crio-c172f59289772a4614e3798828cf11986e8c400c2e220827bca7aee05f04b37f WatchSource:0}: Error finding container c172f59289772a4614e3798828cf11986e8c400c2e220827bca7aee05f04b37f: Status 404 returned error can't find the container with id c172f59289772a4614e3798828cf11986e8c400c2e220827bca7aee05f04b37f
Jul 14 17:10:09.085179 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:09.085149    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7" event={"ID":"88fee8f0-8236-4d6a-95aa-1355865b465a","Type":"ContainerStarted","Data":"c172f59289772a4614e3798828cf11986e8c400c2e220827bca7aee05f04b37f"}
Jul 14 17:10:12.097563 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:12.097527    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7" event={"ID":"88fee8f0-8236-4d6a-95aa-1355865b465a","Type":"ContainerStarted","Data":"58c3b092218091f436b019b96266e3f16db11e4352e7d2df2493afe877e94e42"}
Jul 14 17:10:13.102552 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:13.102517    2580 generic.go:356] "Generic (PLEG): container finished" podID="88fee8f0-8236-4d6a-95aa-1355865b465a" containerID="58c3b092218091f436b019b96266e3f16db11e4352e7d2df2493afe877e94e42" exitCode=0
Jul 14 17:10:13.102552 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:13.102553    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7" event={"ID":"88fee8f0-8236-4d6a-95aa-1355865b465a","Type":"ContainerDied","Data":"58c3b092218091f436b019b96266e3f16db11e4352e7d2df2493afe877e94e42"}
Jul 14 17:10:13.672136 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:13.671693    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:10:13.672136 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:13.672067    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:10:13.675010 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:13.674990    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 14 17:10:14.107632 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:14.107590    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7" event={"ID":"88fee8f0-8236-4d6a-95aa-1355865b465a","Type":"ContainerStarted","Data":"48e385100c884cb8fc160a5216f29395dfac0d51678f6d06d3696d5bb3d41e3c"}
Jul 14 17:10:14.109976 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:14.107733    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:14.152670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:14.152617    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7" podStartSLOduration=2.401563094 podStartE2EDuration="7.152599376s" totalImagesPullingTime="3.681817421s" totalInitContainerRuntime="1.069218861s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:14.150051134 +0000 UTC m=+300.897651031" watchObservedRunningTime="2026-07-14 17:10:14.152599376 +0000 UTC m=+300.900199253"
Jul 14 17:10:25.113986 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:25.113955    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-k7fk7"
Jul 14 17:10:27.432529 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.432498    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"]
Jul 14 17:10:27.442730 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.442701    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.445571 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.445549    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 14 17:10:27.446036 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.446016    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 14 17:10:27.446766 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.446748    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 14 17:10:27.446766 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.446755    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-645gb\""
Jul 14 17:10:27.446946 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.446783    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 17:10:27.452306 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.452284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"]
Jul 14 17:10:27.514160 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.514129    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cg4vk"]
Jul 14 17:10:27.516310 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.516293    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:27.518946 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.518921    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 14 17:10:27.519081 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.519011    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-d5p7l\""
Jul 14 17:10:27.529462 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.529439    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cg4vk"]
Jul 14 17:10:27.555233 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.555205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e40ced1-e25e-4f30-a89e-5bc71341474c-tmp\") pod \"kube-auth-proxy-5c548b7c75-4pfqj\" (UID: \"1e40ced1-e25e-4f30-a89e-5bc71341474c\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.555388 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.555245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgnc8\" (UniqueName: \"kubernetes.io/projected/1e40ced1-e25e-4f30-a89e-5bc71341474c-kube-api-access-qgnc8\") pod \"kube-auth-proxy-5c548b7c75-4pfqj\" (UID: \"1e40ced1-e25e-4f30-a89e-5bc71341474c\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.555388 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.555316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e40ced1-e25e-4f30-a89e-5bc71341474c-tls-certs\") pod \"kube-auth-proxy-5c548b7c75-4pfqj\" (UID: \"1e40ced1-e25e-4f30-a89e-5bc71341474c\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.555388 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.555375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6507a330-6394-495b-b252-d73fc30b83f1-cert\") pod \"odh-model-controller-7998c67dc4-cg4vk\" (UID: \"6507a330-6394-495b-b252-d73fc30b83f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:27.555500 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.555399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k22jq\" (UniqueName: \"kubernetes.io/projected/6507a330-6394-495b-b252-d73fc30b83f1-kube-api-access-k22jq\") pod \"odh-model-controller-7998c67dc4-cg4vk\" (UID: \"6507a330-6394-495b-b252-d73fc30b83f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:27.656163 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.656125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e40ced1-e25e-4f30-a89e-5bc71341474c-tls-certs\") pod \"kube-auth-proxy-5c548b7c75-4pfqj\" (UID: \"1e40ced1-e25e-4f30-a89e-5bc71341474c\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.656163 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.656168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6507a330-6394-495b-b252-d73fc30b83f1-cert\") pod \"odh-model-controller-7998c67dc4-cg4vk\" (UID: \"6507a330-6394-495b-b252-d73fc30b83f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:27.656369 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.656192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k22jq\" (UniqueName: \"kubernetes.io/projected/6507a330-6394-495b-b252-d73fc30b83f1-kube-api-access-k22jq\") pod \"odh-model-controller-7998c67dc4-cg4vk\" (UID: \"6507a330-6394-495b-b252-d73fc30b83f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:27.656369 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.656228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e40ced1-e25e-4f30-a89e-5bc71341474c-tmp\") pod \"kube-auth-proxy-5c548b7c75-4pfqj\" (UID: \"1e40ced1-e25e-4f30-a89e-5bc71341474c\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.656369 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.656247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgnc8\" (UniqueName: \"kubernetes.io/projected/1e40ced1-e25e-4f30-a89e-5bc71341474c-kube-api-access-qgnc8\") pod \"kube-auth-proxy-5c548b7c75-4pfqj\" (UID: \"1e40ced1-e25e-4f30-a89e-5bc71341474c\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.656369 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:10:27.656295    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 14 17:10:27.656369 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:10:27.656368    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6507a330-6394-495b-b252-d73fc30b83f1-cert podName:6507a330-6394-495b-b252-d73fc30b83f1 nodeName:}" failed. No retries permitted until 2026-07-14 17:10:28.156347807 +0000 UTC m=+314.903947665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6507a330-6394-495b-b252-d73fc30b83f1-cert") pod "odh-model-controller-7998c67dc4-cg4vk" (UID: "6507a330-6394-495b-b252-d73fc30b83f1") : secret "odh-model-controller-webhook-cert" not found
Jul 14 17:10:27.658681 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.658654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e40ced1-e25e-4f30-a89e-5bc71341474c-tmp\") pod \"kube-auth-proxy-5c548b7c75-4pfqj\" (UID: \"1e40ced1-e25e-4f30-a89e-5bc71341474c\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.658892 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.658875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e40ced1-e25e-4f30-a89e-5bc71341474c-tls-certs\") pod \"kube-auth-proxy-5c548b7c75-4pfqj\" (UID: \"1e40ced1-e25e-4f30-a89e-5bc71341474c\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.667785 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.667755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k22jq\" (UniqueName: \"kubernetes.io/projected/6507a330-6394-495b-b252-d73fc30b83f1-kube-api-access-k22jq\") pod \"odh-model-controller-7998c67dc4-cg4vk\" (UID: \"6507a330-6394-495b-b252-d73fc30b83f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:27.667785 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.667776    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgnc8\" (UniqueName: \"kubernetes.io/projected/1e40ced1-e25e-4f30-a89e-5bc71341474c-kube-api-access-qgnc8\") pod \"kube-auth-proxy-5c548b7c75-4pfqj\" (UID: \"1e40ced1-e25e-4f30-a89e-5bc71341474c\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.752716 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.752682    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"
Jul 14 17:10:27.899996 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.899943    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj"]
Jul 14 17:10:27.902518 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:10:27.902490    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e40ced1_e25e_4f30_a89e_5bc71341474c.slice/crio-e2b354bb96d9a2990de319a57ffaba5314cfedc4045a23c21b4cfcbe50595681 WatchSource:0}: Error finding container e2b354bb96d9a2990de319a57ffaba5314cfedc4045a23c21b4cfcbe50595681: Status 404 returned error can't find the container with id e2b354bb96d9a2990de319a57ffaba5314cfedc4045a23c21b4cfcbe50595681
Jul 14 17:10:27.904248 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:27.904231    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 17:10:28.153188 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.153105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj" event={"ID":"1e40ced1-e25e-4f30-a89e-5bc71341474c","Type":"ContainerStarted","Data":"e2b354bb96d9a2990de319a57ffaba5314cfedc4045a23c21b4cfcbe50595681"}
Jul 14 17:10:28.162534 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.162505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6507a330-6394-495b-b252-d73fc30b83f1-cert\") pod \"odh-model-controller-7998c67dc4-cg4vk\" (UID: \"6507a330-6394-495b-b252-d73fc30b83f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:28.165101 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.165076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6507a330-6394-495b-b252-d73fc30b83f1-cert\") pod \"odh-model-controller-7998c67dc4-cg4vk\" (UID: \"6507a330-6394-495b-b252-d73fc30b83f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:28.426918 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.426819    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:28.478974 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.478938    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n5z6s"]
Jul 14 17:10:28.481620 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.481597    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:28.484896 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:10:28.484814    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"model-serving-api-tls\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" type="*v1.Secret"
Jul 14 17:10:28.485042 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:10:28.484918    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"model-serving-api-54d89bd5fc-n5z6s\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="21143ebd-b32e-4e43-9584-1fec4ff6e44d" pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:28.485165 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:10:28.485140    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"model-serving-api-dockercfg-z7vjx\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-z7vjx\"" type="*v1.Secret"
Jul 14 17:10:28.497442 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.497413    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n5z6s"]
Jul 14 17:10:28.566172 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.566130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21143ebd-b32e-4e43-9584-1fec4ff6e44d-tls-certs\") pod \"model-serving-api-54d89bd5fc-n5z6s\" (UID: \"21143ebd-b32e-4e43-9584-1fec4ff6e44d\") " pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:28.566172 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.566176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-554dp\" (UniqueName: \"kubernetes.io/projected/21143ebd-b32e-4e43-9584-1fec4ff6e44d-kube-api-access-554dp\") pod \"model-serving-api-54d89bd5fc-n5z6s\" (UID: \"21143ebd-b32e-4e43-9584-1fec4ff6e44d\") " pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:28.567485 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.567463    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cg4vk"]
Jul 14 17:10:28.569081 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:10:28.569050    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6507a330_6394_495b_b252_d73fc30b83f1.slice/crio-cd48fc4cd3e7e14221213c0e42e47e40f231de33ec1f9b4547721a04a491c5a8 WatchSource:0}: Error finding container cd48fc4cd3e7e14221213c0e42e47e40f231de33ec1f9b4547721a04a491c5a8: Status 404 returned error can't find the container with id cd48fc4cd3e7e14221213c0e42e47e40f231de33ec1f9b4547721a04a491c5a8
Jul 14 17:10:28.631921 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.631886    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"]
Jul 14 17:10:28.634037 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.634018    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.636989 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.636805    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 14 17:10:28.636989 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.636806    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 14 17:10:28.636989 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.636882    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6wdsg\""
Jul 14 17:10:28.637236 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.637162    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 14 17:10:28.637297 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.637262    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 14 17:10:28.652703 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.652680    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"]
Jul 14 17:10:28.666793 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.666762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21143ebd-b32e-4e43-9584-1fec4ff6e44d-tls-certs\") pod \"model-serving-api-54d89bd5fc-n5z6s\" (UID: \"21143ebd-b32e-4e43-9584-1fec4ff6e44d\") " pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:28.666970 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.666820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-554dp\" (UniqueName: \"kubernetes.io/projected/21143ebd-b32e-4e43-9584-1fec4ff6e44d-kube-api-access-554dp\") pod \"model-serving-api-54d89bd5fc-n5z6s\" (UID: \"21143ebd-b32e-4e43-9584-1fec4ff6e44d\") " pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:28.679562 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.679495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-554dp\" (UniqueName: \"kubernetes.io/projected/21143ebd-b32e-4e43-9584-1fec4ff6e44d-kube-api-access-554dp\") pod \"model-serving-api-54d89bd5fc-n5z6s\" (UID: \"21143ebd-b32e-4e43-9584-1fec4ff6e44d\") " pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:28.768017 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.767982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-token\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.768017 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.768019    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.768219 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.768047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/114a4503-6ec0-4b56-b3c3-5410818b2b43-local-certs\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.768219 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.768132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68tzf\" (UniqueName: \"kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-kube-api-access-68tzf\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.768219 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.768173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.768219 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.768200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.768219 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.768218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-cacerts\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.868912 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.868872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-cacerts\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.869094 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.868947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-token\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.869094 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.868980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.869094 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.869014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/114a4503-6ec0-4b56-b3c3-5410818b2b43-local-certs\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.869094 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.869054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68tzf\" (UniqueName: \"kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-kube-api-access-68tzf\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.869288 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.869178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.869288 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.869235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.869706 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.869677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.871350 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.871330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/114a4503-6ec0-4b56-b3c3-5410818b2b43-local-certs\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.871531 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.871512    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-cacerts\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.872015 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.871994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.872074 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.872015    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.879819 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.879797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68tzf\" (UniqueName: \"kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-kube-api-access-68tzf\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.880101 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.880081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-token\") pod \"istiod-openshift-gateway-7574459656-pntdd\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:28.945514 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:28.945488    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:29.084879 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:29.084832    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"]
Jul 14 17:10:29.087100 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:10:29.087077    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114a4503_6ec0_4b56_b3c3_5410818b2b43.slice/crio-d2bc1ff8281cf9ae357deaa058fa511c422dc97d520e3c92be40ecf054affdf6 WatchSource:0}: Error finding container d2bc1ff8281cf9ae357deaa058fa511c422dc97d520e3c92be40ecf054affdf6: Status 404 returned error can't find the container with id d2bc1ff8281cf9ae357deaa058fa511c422dc97d520e3c92be40ecf054affdf6
Jul 14 17:10:29.159233 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:29.159190    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk" event={"ID":"6507a330-6394-495b-b252-d73fc30b83f1","Type":"ContainerStarted","Data":"cd48fc4cd3e7e14221213c0e42e47e40f231de33ec1f9b4547721a04a491c5a8"}
Jul 14 17:10:29.160627 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:29.160593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd" event={"ID":"114a4503-6ec0-4b56-b3c3-5410818b2b43","Type":"ContainerStarted","Data":"d2bc1ff8281cf9ae357deaa058fa511c422dc97d520e3c92be40ecf054affdf6"}
Jul 14 17:10:29.667288 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:10:29.667251    2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: failed to sync secret cache: timed out waiting for the condition
Jul 14 17:10:29.667791 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:10:29.667369    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21143ebd-b32e-4e43-9584-1fec4ff6e44d-tls-certs podName:21143ebd-b32e-4e43-9584-1fec4ff6e44d nodeName:}" failed. No retries permitted until 2026-07-14 17:10:30.167344202 +0000 UTC m=+316.914944069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/21143ebd-b32e-4e43-9584-1fec4ff6e44d-tls-certs") pod "model-serving-api-54d89bd5fc-n5z6s" (UID: "21143ebd-b32e-4e43-9584-1fec4ff6e44d") : failed to sync secret cache: timed out waiting for the condition
Jul 14 17:10:29.783021 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:29.782977    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 14 17:10:29.996024 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:29.995992    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-z7vjx\""
Jul 14 17:10:30.179945 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:30.179897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21143ebd-b32e-4e43-9584-1fec4ff6e44d-tls-certs\") pod \"model-serving-api-54d89bd5fc-n5z6s\" (UID: \"21143ebd-b32e-4e43-9584-1fec4ff6e44d\") " pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:30.183385 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:30.183358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21143ebd-b32e-4e43-9584-1fec4ff6e44d-tls-certs\") pod \"model-serving-api-54d89bd5fc-n5z6s\" (UID: \"21143ebd-b32e-4e43-9584-1fec4ff6e44d\") " pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:30.295071 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:30.294993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:33.422830 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:33.422735    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:10:33.422830 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:33.422821    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:10:33.586127 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:33.586083    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n5z6s"]
Jul 14 17:10:33.588670 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:10:33.588634    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21143ebd_b32e_4e43_9584_1fec4ff6e44d.slice/crio-ac38d00a5ad736521ded1cc545c38e4389d0d76a24e4dd37cfd054bf03b22b9e WatchSource:0}: Error finding container ac38d00a5ad736521ded1cc545c38e4389d0d76a24e4dd37cfd054bf03b22b9e: Status 404 returned error can't find the container with id ac38d00a5ad736521ded1cc545c38e4389d0d76a24e4dd37cfd054bf03b22b9e
Jul 14 17:10:34.185339 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.185287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk" event={"ID":"6507a330-6394-495b-b252-d73fc30b83f1","Type":"ContainerStarted","Data":"891bef63546cd12e53c41aae1e5ad828101937e8146b8d51f9d66b0c892da217"}
Jul 14 17:10:34.185669 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.185636    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:34.187332 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.187302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj" event={"ID":"1e40ced1-e25e-4f30-a89e-5bc71341474c","Type":"ContainerStarted","Data":"f103ae0a7f9be141bb8ba279b3a6247c7aeeb88983d542220dd9756b5208e16e"}
Jul 14 17:10:34.189194 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.189148    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd" event={"ID":"114a4503-6ec0-4b56-b3c3-5410818b2b43","Type":"ContainerStarted","Data":"7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54"}
Jul 14 17:10:34.189303 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.189259    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:34.191349 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.191300    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s" event={"ID":"21143ebd-b32e-4e43-9584-1fec4ff6e44d","Type":"ContainerStarted","Data":"ac38d00a5ad736521ded1cc545c38e4389d0d76a24e4dd37cfd054bf03b22b9e"}
Jul 14 17:10:34.191503 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.191357    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-pntdd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 14 17:10:34.191503 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.191416    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd" podUID="114a4503-6ec0-4b56-b3c3-5410818b2b43" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 17:10:34.215517 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.215469    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk" podStartSLOduration=2.369457292 podStartE2EDuration="7.215454614s" totalImagesPullingTime="4.845997322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:34.212817239 +0000 UTC m=+320.960417117" watchObservedRunningTime="2026-07-14 17:10:34.215454614 +0000 UTC m=+320.963054509"
Jul 14 17:10:34.247350 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.247300    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd" podStartSLOduration=1.91391015 podStartE2EDuration="6.247282349s" totalImagesPullingTime="4.333372199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:34.245787335 +0000 UTC m=+320.993387230" watchObservedRunningTime="2026-07-14 17:10:34.247282349 +0000 UTC m=+320.994882226"
Jul 14 17:10:34.272377 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:34.272311    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c548b7c75-4pfqj" podStartSLOduration=1.7470875540000002 podStartE2EDuration="7.272288504s" totalImagesPullingTime="5.52520095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:34.271165418 +0000 UTC m=+321.018765297" watchObservedRunningTime="2026-07-14 17:10:34.272288504 +0000 UTC m=+321.019888379"
Jul 14 17:10:35.196213 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:35.196188    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:36.201525 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:36.201493    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s" event={"ID":"21143ebd-b32e-4e43-9584-1fec4ff6e44d","Type":"ContainerStarted","Data":"b205c8dcfa5f89c686b64ad599bafdabfb0bcfe5a423e3aec1c7993842042d40"}
Jul 14 17:10:36.229325 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:36.229273    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s" podStartSLOduration=6.091824158 podStartE2EDuration="8.229255602s" totalImagesPullingTime="2.137431444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:36.227761513 +0000 UTC m=+322.975361390" watchObservedRunningTime="2026-07-14 17:10:36.229255602 +0000 UTC m=+322.976855479"
Jul 14 17:10:37.205391 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:37.205363    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:10:38.186592 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.186557    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"]
Jul 14 17:10:38.188705 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.188688    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:10:38.192105 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.192084    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-krhgn\""
Jul 14 17:10:38.192590 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.192561    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 14 17:10:38.200744 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.200723    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"]
Jul 14 17:10:38.208943 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.208907    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"
Jul 14 17:10:38.210897 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.210874    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"]
Jul 14 17:10:38.212052 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.212020    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-t2km7\""
Jul 14 17:10:38.212420 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.212405    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 14 17:10:38.220923 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.220616    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"]
Jul 14 17:10:38.252993 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.252957    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jlmm\" (UniqueName: \"kubernetes.io/projected/efacff4e-e30c-41bf-ad77-d5b30d0323d8-kube-api-access-9jlmm\") pod \"kserve-controller-manager-d6f4b65b8-sv6wr\" (UID: \"efacff4e-e30c-41bf-ad77-d5b30d0323d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:10:38.253198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.253082    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efacff4e-e30c-41bf-ad77-d5b30d0323d8-cert\") pod \"kserve-controller-manager-d6f4b65b8-sv6wr\" (UID: \"efacff4e-e30c-41bf-ad77-d5b30d0323d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:10:38.354082 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.354045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efacff4e-e30c-41bf-ad77-d5b30d0323d8-cert\") pod \"kserve-controller-manager-d6f4b65b8-sv6wr\" (UID: \"efacff4e-e30c-41bf-ad77-d5b30d0323d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:10:38.354263 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.354091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jlmm\" (UniqueName: \"kubernetes.io/projected/efacff4e-e30c-41bf-ad77-d5b30d0323d8-kube-api-access-9jlmm\") pod \"kserve-controller-manager-d6f4b65b8-sv6wr\" (UID: \"efacff4e-e30c-41bf-ad77-d5b30d0323d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:10:38.354263 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:10:38.354203    2580 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 14 17:10:38.354337 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:10:38.354278    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efacff4e-e30c-41bf-ad77-d5b30d0323d8-cert podName:efacff4e-e30c-41bf-ad77-d5b30d0323d8 nodeName:}" failed. No retries permitted until 2026-07-14 17:10:38.854253781 +0000 UTC m=+325.601853661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/efacff4e-e30c-41bf-ad77-d5b30d0323d8-cert") pod "kserve-controller-manager-d6f4b65b8-sv6wr" (UID: "efacff4e-e30c-41bf-ad77-d5b30d0323d8") : secret "kserve-webhook-server-cert" not found
Jul 14 17:10:38.354384 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.354359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64w7f\" (UniqueName: \"kubernetes.io/projected/6706a086-5283-4c17-815b-f8a38703ab5d-kube-api-access-64w7f\") pod \"llmisvc-controller-manager-777c6bd7d5-gh5jg\" (UID: \"6706a086-5283-4c17-815b-f8a38703ab5d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"
Jul 14 17:10:38.354430 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.354392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6706a086-5283-4c17-815b-f8a38703ab5d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gh5jg\" (UID: \"6706a086-5283-4c17-815b-f8a38703ab5d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"
Jul 14 17:10:38.365170 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.365138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jlmm\" (UniqueName: \"kubernetes.io/projected/efacff4e-e30c-41bf-ad77-d5b30d0323d8-kube-api-access-9jlmm\") pod \"kserve-controller-manager-d6f4b65b8-sv6wr\" (UID: \"efacff4e-e30c-41bf-ad77-d5b30d0323d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:10:38.454916 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.454892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64w7f\" (UniqueName: \"kubernetes.io/projected/6706a086-5283-4c17-815b-f8a38703ab5d-kube-api-access-64w7f\") pod \"llmisvc-controller-manager-777c6bd7d5-gh5jg\" (UID: \"6706a086-5283-4c17-815b-f8a38703ab5d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"
Jul 14 17:10:38.455078 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.454928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6706a086-5283-4c17-815b-f8a38703ab5d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gh5jg\" (UID: \"6706a086-5283-4c17-815b-f8a38703ab5d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"
Jul 14 17:10:38.457537 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.457514    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6706a086-5283-4c17-815b-f8a38703ab5d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gh5jg\" (UID: \"6706a086-5283-4c17-815b-f8a38703ab5d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"
Jul 14 17:10:38.466885 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.466862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64w7f\" (UniqueName: \"kubernetes.io/projected/6706a086-5283-4c17-815b-f8a38703ab5d-kube-api-access-64w7f\") pod \"llmisvc-controller-manager-777c6bd7d5-gh5jg\" (UID: \"6706a086-5283-4c17-815b-f8a38703ab5d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"
Jul 14 17:10:38.521748 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.521712    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"
Jul 14 17:10:38.659918 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.659900    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"]
Jul 14 17:10:38.661954 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:10:38.661924    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6706a086_5283_4c17_815b_f8a38703ab5d.slice/crio-c29f4ec0c616b90109214af62b22c6662fbf3cac821c69b29a50c20420e84fad WatchSource:0}: Error finding container c29f4ec0c616b90109214af62b22c6662fbf3cac821c69b29a50c20420e84fad: Status 404 returned error can't find the container with id c29f4ec0c616b90109214af62b22c6662fbf3cac821c69b29a50c20420e84fad
Jul 14 17:10:38.858461 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.858384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efacff4e-e30c-41bf-ad77-d5b30d0323d8-cert\") pod \"kserve-controller-manager-d6f4b65b8-sv6wr\" (UID: \"efacff4e-e30c-41bf-ad77-d5b30d0323d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:10:38.860901 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:38.860882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efacff4e-e30c-41bf-ad77-d5b30d0323d8-cert\") pod \"kserve-controller-manager-d6f4b65b8-sv6wr\" (UID: \"efacff4e-e30c-41bf-ad77-d5b30d0323d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:10:39.100188 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:39.100150    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:10:39.217240 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:39.217189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg" event={"ID":"6706a086-5283-4c17-815b-f8a38703ab5d","Type":"ContainerStarted","Data":"c29f4ec0c616b90109214af62b22c6662fbf3cac821c69b29a50c20420e84fad"}
Jul 14 17:10:39.233638 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:39.233608    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"]
Jul 14 17:10:39.236593 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:10:39.236568    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefacff4e_e30c_41bf_ad77_d5b30d0323d8.slice/crio-d19a842f583278abca1d103be1625672607f68a1324cea3765ddd98e588def62 WatchSource:0}: Error finding container d19a842f583278abca1d103be1625672607f68a1324cea3765ddd98e588def62: Status 404 returned error can't find the container with id d19a842f583278abca1d103be1625672607f68a1324cea3765ddd98e588def62
Jul 14 17:10:40.225827 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:40.225785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr" event={"ID":"efacff4e-e30c-41bf-ad77-d5b30d0323d8","Type":"ContainerStarted","Data":"d19a842f583278abca1d103be1625672607f68a1324cea3765ddd98e588def62"}
Jul 14 17:10:41.344281 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.344238    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"]
Jul 14 17:10:41.344723 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.344467    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd" podUID="114a4503-6ec0-4b56-b3c3-5410818b2b43" containerName="discovery" containerID="cri-o://7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54" gracePeriod=30
Jul 14 17:10:41.601025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.600958    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:41.684834 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.684796    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/114a4503-6ec0-4b56-b3c3-5410818b2b43-local-certs\") pod \"114a4503-6ec0-4b56-b3c3-5410818b2b43\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") "
Jul 14 17:10:41.684834 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.684836    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-cacerts\") pod \"114a4503-6ec0-4b56-b3c3-5410818b2b43\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") "
Jul 14 17:10:41.685093 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.684912    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-dns-cert\") pod \"114a4503-6ec0-4b56-b3c3-5410818b2b43\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") "
Jul 14 17:10:41.685093 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.684936    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-token\") pod \"114a4503-6ec0-4b56-b3c3-5410818b2b43\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") "
Jul 14 17:10:41.685093 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.684965    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-kubeconfig\") pod \"114a4503-6ec0-4b56-b3c3-5410818b2b43\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") "
Jul 14 17:10:41.685093 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.685043    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-ca-configmap\") pod \"114a4503-6ec0-4b56-b3c3-5410818b2b43\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") "
Jul 14 17:10:41.685093 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.685077    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68tzf\" (UniqueName: \"kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-kube-api-access-68tzf\") pod \"114a4503-6ec0-4b56-b3c3-5410818b2b43\" (UID: \"114a4503-6ec0-4b56-b3c3-5410818b2b43\") "
Jul 14 17:10:41.685754 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.685696    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "114a4503-6ec0-4b56-b3c3-5410818b2b43" (UID: "114a4503-6ec0-4b56-b3c3-5410818b2b43"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:10:41.688070 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.687974    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-cacerts" (OuterVolumeSpecName: "cacerts") pod "114a4503-6ec0-4b56-b3c3-5410818b2b43" (UID: "114a4503-6ec0-4b56-b3c3-5410818b2b43"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:10:41.688070 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.687998    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114a4503-6ec0-4b56-b3c3-5410818b2b43-local-certs" (OuterVolumeSpecName: "local-certs") pod "114a4503-6ec0-4b56-b3c3-5410818b2b43" (UID: "114a4503-6ec0-4b56-b3c3-5410818b2b43"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:10:41.688254 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.688233    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "114a4503-6ec0-4b56-b3c3-5410818b2b43" (UID: "114a4503-6ec0-4b56-b3c3-5410818b2b43"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:10:41.688323 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.688299    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "114a4503-6ec0-4b56-b3c3-5410818b2b43" (UID: "114a4503-6ec0-4b56-b3c3-5410818b2b43"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:10:41.688439 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.688355    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-token" (OuterVolumeSpecName: "istio-token") pod "114a4503-6ec0-4b56-b3c3-5410818b2b43" (UID: "114a4503-6ec0-4b56-b3c3-5410818b2b43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:10:41.688579 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.688556    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-kube-api-access-68tzf" (OuterVolumeSpecName: "kube-api-access-68tzf") pod "114a4503-6ec0-4b56-b3c3-5410818b2b43" (UID: "114a4503-6ec0-4b56-b3c3-5410818b2b43"). InnerVolumeSpecName "kube-api-access-68tzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:10:41.786144 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.786112    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-ca-configmap\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:10:41.786144 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.786146    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68tzf\" (UniqueName: \"kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-kube-api-access-68tzf\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:10:41.786377 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.786161    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/114a4503-6ec0-4b56-b3c3-5410818b2b43-local-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:10:41.786377 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.786174    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-cacerts\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:10:41.786377 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.786186    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-csr-dns-cert\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:10:41.786377 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.786200    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-token\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:10:41.786377 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:41.786212    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/114a4503-6ec0-4b56-b3c3-5410818b2b43-istio-kubeconfig\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:10:42.236411 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:42.236369    2580 generic.go:356] "Generic (PLEG): container finished" podID="114a4503-6ec0-4b56-b3c3-5410818b2b43" containerID="7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54" exitCode=0
Jul 14 17:10:42.236411 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:42.236416    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"
Jul 14 17:10:42.236659 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:42.236463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd" event={"ID":"114a4503-6ec0-4b56-b3c3-5410818b2b43","Type":"ContainerDied","Data":"7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54"}
Jul 14 17:10:42.236659 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:42.236524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pntdd" event={"ID":"114a4503-6ec0-4b56-b3c3-5410818b2b43","Type":"ContainerDied","Data":"d2bc1ff8281cf9ae357deaa058fa511c422dc97d520e3c92be40ecf054affdf6"}
Jul 14 17:10:42.236659 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:42.236554    2580 scope.go:117] "RemoveContainer" containerID="7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54"
Jul 14 17:10:42.288067 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:42.288034    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"]
Jul 14 17:10:42.293424 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:42.293400    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pntdd"]
Jul 14 17:10:42.361680 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:42.361648    2580 scope.go:117] "RemoveContainer" containerID="7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54"
Jul 14 17:10:42.362102 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:10:42.362076    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54\": container with ID starting with 7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54 not found: ID does not exist" containerID="7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54"
Jul 14 17:10:42.362177 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:42.362118    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54"} err="failed to get container status \"7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54\": rpc error: code = NotFound desc = could not find container \"7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54\": container with ID starting with 7afbec59d8433b1a76d916c44b330d62b20a6abcd352fe04d68f287487c3fe54 not found: ID does not exist"
Jul 14 17:10:43.241698 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:43.241654    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg" event={"ID":"6706a086-5283-4c17-815b-f8a38703ab5d","Type":"ContainerStarted","Data":"1d3e41eba9836bd9db38a6ba0bd21a55ecab5261a5393406c7204b539e30c8e9"}
Jul 14 17:10:43.241909 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:43.241793    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"
Jul 14 17:10:43.243232 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:43.243210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr" event={"ID":"efacff4e-e30c-41bf-ad77-d5b30d0323d8","Type":"ContainerStarted","Data":"d4bde334a119f41cb8c315d480ecaea2ce2daac82767311cff330a11e141d3a6"}
Jul 14 17:10:43.243327 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:43.243259    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:10:43.261019 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:43.260976    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg" podStartSLOduration=1.001337318 podStartE2EDuration="5.260960631s" totalImagesPullingTime="4.259623313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:43.260213139 +0000 UTC m=+330.007813036" watchObservedRunningTime="2026-07-14 17:10:43.260960631 +0000 UTC m=+330.008560507"
Jul 14 17:10:43.285063 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:43.285013    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr" podStartSLOduration=1.602403537 podStartE2EDuration="5.284998126s" totalImagesPullingTime="3.682594589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:43.283795578 +0000 UTC m=+330.031395475" watchObservedRunningTime="2026-07-14 17:10:43.284998126 +0000 UTC m=+330.032598002"
Jul 14 17:10:43.881303 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:43.881269    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="114a4503-6ec0-4b56-b3c3-5410818b2b43" path="/var/lib/kubelet/pods/114a4503-6ec0-4b56-b3c3-5410818b2b43/volumes"
Jul 14 17:10:45.198280 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:45.198242    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-cg4vk"
Jul 14 17:10:48.216893 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:10:48.216861    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-n5z6s"
Jul 14 17:11:14.250349 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:14.250320    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh5jg"
Jul 14 17:11:14.253709 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:14.253690    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv6wr"
Jul 14 17:11:28.771338 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.771301    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tgxtx"]
Jul 14 17:11:28.771707 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.771582    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="114a4503-6ec0-4b56-b3c3-5410818b2b43" containerName="discovery"
Jul 14 17:11:28.771707 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.771592    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="114a4503-6ec0-4b56-b3c3-5410818b2b43" containerName="discovery"
Jul 14 17:11:28.771707 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.771643    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="114a4503-6ec0-4b56-b3c3-5410818b2b43" containerName="discovery"
Jul 14 17:11:28.781259 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.781233    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tgxtx"
Jul 14 17:11:28.783814 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.783789    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2hcdx\""
Jul 14 17:11:28.783814 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.783810    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 17:11:28.785005 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.784986    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 17:11:28.788024 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.788002    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tgxtx"]
Jul 14 17:11:28.844248 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.844208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg68r\" (UniqueName: \"kubernetes.io/projected/dd5a726a-1e37-48cc-b5c9-51e7491daadc-kube-api-access-mg68r\") pod \"kuadrant-operator-catalog-tgxtx\" (UID: \"dd5a726a-1e37-48cc-b5c9-51e7491daadc\") " pod="kuadrant-system/kuadrant-operator-catalog-tgxtx"
Jul 14 17:11:28.944990 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.944943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg68r\" (UniqueName: \"kubernetes.io/projected/dd5a726a-1e37-48cc-b5c9-51e7491daadc-kube-api-access-mg68r\") pod \"kuadrant-operator-catalog-tgxtx\" (UID: \"dd5a726a-1e37-48cc-b5c9-51e7491daadc\") " pod="kuadrant-system/kuadrant-operator-catalog-tgxtx"
Jul 14 17:11:28.954818 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:28.954784    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg68r\" (UniqueName: \"kubernetes.io/projected/dd5a726a-1e37-48cc-b5c9-51e7491daadc-kube-api-access-mg68r\") pod \"kuadrant-operator-catalog-tgxtx\" (UID: \"dd5a726a-1e37-48cc-b5c9-51e7491daadc\") " pod="kuadrant-system/kuadrant-operator-catalog-tgxtx"
Jul 14 17:11:29.092395 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.092291    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tgxtx"
Jul 14 17:11:29.124378 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.124342    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tgxtx"]
Jul 14 17:11:29.223581 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.223559    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tgxtx"]
Jul 14 17:11:29.226113 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:11:29.226085    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5a726a_1e37_48cc_b5c9_51e7491daadc.slice/crio-4d127e98434df8281c51c4c4d478661c7ca307cefd3e5507b53cc632cbfc531e WatchSource:0}: Error finding container 4d127e98434df8281c51c4c4d478661c7ca307cefd3e5507b53cc632cbfc531e: Status 404 returned error can't find the container with id 4d127e98434df8281c51c4c4d478661c7ca307cefd3e5507b53cc632cbfc531e
Jul 14 17:11:29.335793 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.335754    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-clqr2"]
Jul 14 17:11:29.338491 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.338473    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-clqr2"
Jul 14 17:11:29.347540 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.347500    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-clqr2"]
Jul 14 17:11:29.428280 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.428244    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tgxtx" event={"ID":"dd5a726a-1e37-48cc-b5c9-51e7491daadc","Type":"ContainerStarted","Data":"4d127e98434df8281c51c4c4d478661c7ca307cefd3e5507b53cc632cbfc531e"}
Jul 14 17:11:29.449630 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.449599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7zmd\" (UniqueName: \"kubernetes.io/projected/5f5c9fa8-58fb-48c9-b3c8-0fa3a0d486cf-kube-api-access-w7zmd\") pod \"kuadrant-operator-catalog-clqr2\" (UID: \"5f5c9fa8-58fb-48c9-b3c8-0fa3a0d486cf\") " pod="kuadrant-system/kuadrant-operator-catalog-clqr2"
Jul 14 17:11:29.550997 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.550956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7zmd\" (UniqueName: \"kubernetes.io/projected/5f5c9fa8-58fb-48c9-b3c8-0fa3a0d486cf-kube-api-access-w7zmd\") pod \"kuadrant-operator-catalog-clqr2\" (UID: \"5f5c9fa8-58fb-48c9-b3c8-0fa3a0d486cf\") " pod="kuadrant-system/kuadrant-operator-catalog-clqr2"
Jul 14 17:11:29.565365 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.565328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7zmd\" (UniqueName: \"kubernetes.io/projected/5f5c9fa8-58fb-48c9-b3c8-0fa3a0d486cf-kube-api-access-w7zmd\") pod \"kuadrant-operator-catalog-clqr2\" (UID: \"5f5c9fa8-58fb-48c9-b3c8-0fa3a0d486cf\") " pod="kuadrant-system/kuadrant-operator-catalog-clqr2"
Jul 14 17:11:29.649235 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.649140    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-clqr2"
Jul 14 17:11:29.790447 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:29.790415    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-clqr2"]
Jul 14 17:11:29.817041 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:11:29.816996    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5c9fa8_58fb_48c9_b3c8_0fa3a0d486cf.slice/crio-57f32a2340611da3fdfebdf09d87993cb1b54108bb31c01d477274ce05c02a2a WatchSource:0}: Error finding container 57f32a2340611da3fdfebdf09d87993cb1b54108bb31c01d477274ce05c02a2a: Status 404 returned error can't find the container with id 57f32a2340611da3fdfebdf09d87993cb1b54108bb31c01d477274ce05c02a2a
Jul 14 17:11:30.434550 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:30.434513    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-clqr2" event={"ID":"5f5c9fa8-58fb-48c9-b3c8-0fa3a0d486cf","Type":"ContainerStarted","Data":"57f32a2340611da3fdfebdf09d87993cb1b54108bb31c01d477274ce05c02a2a"}
Jul 14 17:11:32.444771 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:32.444727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-clqr2" event={"ID":"5f5c9fa8-58fb-48c9-b3c8-0fa3a0d486cf","Type":"ContainerStarted","Data":"7df363f2ea43a7ce05fbbc8d11bc6fa061775a23804e3a11fc2035c8b79ee291"}
Jul 14 17:11:32.446224 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:32.446196    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tgxtx" event={"ID":"dd5a726a-1e37-48cc-b5c9-51e7491daadc","Type":"ContainerStarted","Data":"29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587"}
Jul 14 17:11:32.446350 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:32.446247    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-tgxtx" podUID="dd5a726a-1e37-48cc-b5c9-51e7491daadc" containerName="registry-server" containerID="cri-o://29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587" gracePeriod=2
Jul 14 17:11:32.464348 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:32.464301    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-clqr2" podStartSLOduration=1.7971378489999998 podStartE2EDuration="3.464284427s" totalImagesPullingTime="1.667146578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:11:32.463409616 +0000 UTC m=+379.211009494" watchObservedRunningTime="2026-07-14 17:11:32.464284427 +0000 UTC m=+379.211884313"
Jul 14 17:11:32.481537 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:32.481489    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tgxtx" podStartSLOduration=2.235074254 podStartE2EDuration="4.481473761s" totalImagesPullingTime="2.246399507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:11:32.480390374 +0000 UTC m=+379.227990251" watchObservedRunningTime="2026-07-14 17:11:32.481473761 +0000 UTC m=+379.229073692"
Jul 14 17:11:32.688758 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:32.688731    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tgxtx"
Jul 14 17:11:32.781294 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:32.781185    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg68r\" (UniqueName: \"kubernetes.io/projected/dd5a726a-1e37-48cc-b5c9-51e7491daadc-kube-api-access-mg68r\") pod \"dd5a726a-1e37-48cc-b5c9-51e7491daadc\" (UID: \"dd5a726a-1e37-48cc-b5c9-51e7491daadc\") "
Jul 14 17:11:32.783502 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:32.783470    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd5a726a-1e37-48cc-b5c9-51e7491daadc-kube-api-access-mg68r" (OuterVolumeSpecName: "kube-api-access-mg68r") pod "dd5a726a-1e37-48cc-b5c9-51e7491daadc" (UID: "dd5a726a-1e37-48cc-b5c9-51e7491daadc"). InnerVolumeSpecName "kube-api-access-mg68r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:11:32.882145 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:32.882103    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg68r\" (UniqueName: \"kubernetes.io/projected/dd5a726a-1e37-48cc-b5c9-51e7491daadc-kube-api-access-mg68r\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:11:33.451679 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:33.451637    2580 generic.go:356] "Generic (PLEG): container finished" podID="dd5a726a-1e37-48cc-b5c9-51e7491daadc" containerID="29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587" exitCode=0
Jul 14 17:11:33.452106 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:33.451698    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tgxtx"
Jul 14 17:11:33.452106 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:33.451726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tgxtx" event={"ID":"dd5a726a-1e37-48cc-b5c9-51e7491daadc","Type":"ContainerDied","Data":"29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587"}
Jul 14 17:11:33.452106 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:33.451800    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tgxtx" event={"ID":"dd5a726a-1e37-48cc-b5c9-51e7491daadc","Type":"ContainerDied","Data":"4d127e98434df8281c51c4c4d478661c7ca307cefd3e5507b53cc632cbfc531e"}
Jul 14 17:11:33.452106 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:33.451823    2580 scope.go:117] "RemoveContainer" containerID="29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587"
Jul 14 17:11:33.460540 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:33.460519    2580 scope.go:117] "RemoveContainer" containerID="29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587"
Jul 14 17:11:33.460799 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:11:33.460780    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587\": container with ID starting with 29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587 not found: ID does not exist" containerID="29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587"
Jul 14 17:11:33.460883 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:33.460806    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587"} err="failed to get container status \"29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587\": rpc error: code = NotFound desc = could not find container \"29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587\": container with ID starting with 29da83cb9f0d4dd509e605389a8825adfd129489903d05bd5d6cf8b926926587 not found: ID does not exist"
Jul 14 17:11:33.473490 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:33.473462    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tgxtx"]
Jul 14 17:11:33.477682 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:33.477660    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tgxtx"]
Jul 14 17:11:33.881856 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:33.881761    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd5a726a-1e37-48cc-b5c9-51e7491daadc" path="/var/lib/kubelet/pods/dd5a726a-1e37-48cc-b5c9-51e7491daadc/volumes"
Jul 14 17:11:39.649563 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:39.649512    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-clqr2"
Jul 14 17:11:39.650014 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:39.649619    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-clqr2"
Jul 14 17:11:39.672695 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:39.672663    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-clqr2"
Jul 14 17:11:40.501199 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:40.501162    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-clqr2"
Jul 14 17:11:42.193105 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.193078    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-545bcf8cf8-hzhsz"]
Jul 14 17:11:42.193510 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.193370    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd5a726a-1e37-48cc-b5c9-51e7491daadc" containerName="registry-server"
Jul 14 17:11:42.193510 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.193381    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd5a726a-1e37-48cc-b5c9-51e7491daadc" containerName="registry-server"
Jul 14 17:11:42.193510 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.193439    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd5a726a-1e37-48cc-b5c9-51e7491daadc" containerName="registry-server"
Jul 14 17:11:42.196349 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.196332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.198911 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.198889    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 14 17:11:42.199039 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.198955    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 14 17:11:42.200240 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.200216    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 14 17:11:42.200342 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.200222    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 14 17:11:42.200402 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.200348    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 14 17:11:42.200402 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.200353    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jkhdp\""
Jul 14 17:11:42.201050 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.201034    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 14 17:11:42.201431 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.201416    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 14 17:11:42.204600 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.204578    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 14 17:11:42.208482 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.208461    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-545bcf8cf8-hzhsz"]
Jul 14 17:11:42.257905 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.257839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d22affd-6642-420f-89c6-61d3bb94b05c-console-oauth-config\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.257905 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.257904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9njn\" (UniqueName: \"kubernetes.io/projected/3d22affd-6642-420f-89c6-61d3bb94b05c-kube-api-access-q9njn\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.258138 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.257962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-trusted-ca-bundle\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.258138 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.258001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-oauth-serving-cert\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.258138 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.258050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-console-config\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.258138 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.258066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-service-ca\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.258138 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.258111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d22affd-6642-420f-89c6-61d3bb94b05c-console-serving-cert\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.359049 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.358999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-console-config\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.359049 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.359049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-service-ca\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.359301 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.359091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d22affd-6642-420f-89c6-61d3bb94b05c-console-serving-cert\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.359301 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.359142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d22affd-6642-420f-89c6-61d3bb94b05c-console-oauth-config\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.359301 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.359165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9njn\" (UniqueName: \"kubernetes.io/projected/3d22affd-6642-420f-89c6-61d3bb94b05c-kube-api-access-q9njn\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.359301 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.359191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-trusted-ca-bundle\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.359301 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.359228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-oauth-serving-cert\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.359827 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.359803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-service-ca\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.359953 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.359940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-console-config\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.360000 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.359947    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-oauth-serving-cert\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.360109 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.360089    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d22affd-6642-420f-89c6-61d3bb94b05c-trusted-ca-bundle\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.362306 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.362284    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d22affd-6642-420f-89c6-61d3bb94b05c-console-serving-cert\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.362418 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.362374    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d22affd-6642-420f-89c6-61d3bb94b05c-console-oauth-config\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.369563 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.369540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9njn\" (UniqueName: \"kubernetes.io/projected/3d22affd-6642-420f-89c6-61d3bb94b05c-kube-api-access-q9njn\") pod \"console-545bcf8cf8-hzhsz\" (UID: \"3d22affd-6642-420f-89c6-61d3bb94b05c\") " pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.505945 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.505908    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:42.639741 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:42.639713    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-545bcf8cf8-hzhsz"]
Jul 14 17:11:42.642199 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:11:42.642171    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d22affd_6642_420f_89c6_61d3bb94b05c.slice/crio-5d016376bf38d59c5518946aa239c805ee9076773b8334923a38c27462bc3197 WatchSource:0}: Error finding container 5d016376bf38d59c5518946aa239c805ee9076773b8334923a38c27462bc3197: Status 404 returned error can't find the container with id 5d016376bf38d59c5518946aa239c805ee9076773b8334923a38c27462bc3197
Jul 14 17:11:43.492173 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:43.492135    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-545bcf8cf8-hzhsz" event={"ID":"3d22affd-6642-420f-89c6-61d3bb94b05c","Type":"ContainerStarted","Data":"5d016376bf38d59c5518946aa239c805ee9076773b8334923a38c27462bc3197"}
Jul 14 17:11:46.505981 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:46.505939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-545bcf8cf8-hzhsz" event={"ID":"3d22affd-6642-420f-89c6-61d3bb94b05c","Type":"ContainerStarted","Data":"d6308a7383bb6ec2fb8e781620d69fa882dea16327197f311b56352f3fea73dc"}
Jul 14 17:11:46.527023 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:46.526974    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-545bcf8cf8-hzhsz" podStartSLOduration=1.004876761 podStartE2EDuration="4.526959045s" totalImagesPullingTime="3.522082284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:11:46.526203248 +0000 UTC m=+393.273803153" watchObservedRunningTime="2026-07-14 17:11:46.526959045 +0000 UTC m=+393.274558923"
Jul 14 17:11:52.506359 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:52.506319    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:52.506948 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:52.506391    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:52.511115 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:52.511089    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:52.535138 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:52.534660    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-545bcf8cf8-hzhsz"
Jul 14 17:11:56.399751 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.399717    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"]
Jul 14 17:11:56.401972 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.401954    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:11:56.404546 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.404527    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mlgp9\""
Jul 14 17:11:56.417092 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.417068    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"]
Jul 14 17:11:56.483142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.483107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28fe39f8-a48d-4a4b-98cc-53012300aaef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" (UID: \"28fe39f8-a48d-4a4b-98cc-53012300aaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:11:56.483332 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.483157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6484j\" (UniqueName: \"kubernetes.io/projected/28fe39f8-a48d-4a4b-98cc-53012300aaef-kube-api-access-6484j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" (UID: \"28fe39f8-a48d-4a4b-98cc-53012300aaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:11:56.583579 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.583537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28fe39f8-a48d-4a4b-98cc-53012300aaef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" (UID: \"28fe39f8-a48d-4a4b-98cc-53012300aaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:11:56.583758 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.583598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6484j\" (UniqueName: \"kubernetes.io/projected/28fe39f8-a48d-4a4b-98cc-53012300aaef-kube-api-access-6484j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" (UID: \"28fe39f8-a48d-4a4b-98cc-53012300aaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:11:56.583975 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.583955    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28fe39f8-a48d-4a4b-98cc-53012300aaef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" (UID: \"28fe39f8-a48d-4a4b-98cc-53012300aaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:11:56.599301 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.599275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6484j\" (UniqueName: \"kubernetes.io/projected/28fe39f8-a48d-4a4b-98cc-53012300aaef-kube-api-access-6484j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" (UID: \"28fe39f8-a48d-4a4b-98cc-53012300aaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:11:56.711994 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.711956    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:11:56.846885 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:56.846861    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"]
Jul 14 17:11:56.848869 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:11:56.848823    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28fe39f8_a48d_4a4b_98cc_53012300aaef.slice/crio-212584b3434fb34c542f9da81ced6436b6c923e2ecee03e7e92cf3a7c1d258dd WatchSource:0}: Error finding container 212584b3434fb34c542f9da81ced6436b6c923e2ecee03e7e92cf3a7c1d258dd: Status 404 returned error can't find the container with id 212584b3434fb34c542f9da81ced6436b6c923e2ecee03e7e92cf3a7c1d258dd
Jul 14 17:11:57.550993 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:57.550952    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf" event={"ID":"28fe39f8-a48d-4a4b-98cc-53012300aaef","Type":"ContainerStarted","Data":"212584b3434fb34c542f9da81ced6436b6c923e2ecee03e7e92cf3a7c1d258dd"}
Jul 14 17:11:58.614621 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:58.614584    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf"]
Jul 14 17:11:58.617059 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:58.617043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf"
Jul 14 17:11:58.619781 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:58.619758    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-946zf\""
Jul 14 17:11:58.619914 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:58.619762    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 14 17:11:58.630715 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:58.630695    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf"]
Jul 14 17:11:58.702239 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:58.702199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6vsd\" (UniqueName: \"kubernetes.io/projected/18bee717-e3de-45f7-a7e2-da8e14181a46-kube-api-access-r6vsd\") pod \"dns-operator-controller-manager-864dd78c4c-sqltf\" (UID: \"18bee717-e3de-45f7-a7e2-da8e14181a46\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf"
Jul 14 17:11:58.802915 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:58.802879    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6vsd\" (UniqueName: \"kubernetes.io/projected/18bee717-e3de-45f7-a7e2-da8e14181a46-kube-api-access-r6vsd\") pod \"dns-operator-controller-manager-864dd78c4c-sqltf\" (UID: \"18bee717-e3de-45f7-a7e2-da8e14181a46\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf"
Jul 14 17:11:58.816120 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:58.816085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6vsd\" (UniqueName: \"kubernetes.io/projected/18bee717-e3de-45f7-a7e2-da8e14181a46-kube-api-access-r6vsd\") pod \"dns-operator-controller-manager-864dd78c4c-sqltf\" (UID: \"18bee717-e3de-45f7-a7e2-da8e14181a46\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf"
Jul 14 17:11:58.927560 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:58.927457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf"
Jul 14 17:11:59.071015 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:59.070992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf"]
Jul 14 17:11:59.073456 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:11:59.073430    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18bee717_e3de_45f7_a7e2_da8e14181a46.slice/crio-f2de71f294ea4af8465983d8deadfd642f96878791924cddd0687eafaa0fcf26 WatchSource:0}: Error finding container f2de71f294ea4af8465983d8deadfd642f96878791924cddd0687eafaa0fcf26: Status 404 returned error can't find the container with id f2de71f294ea4af8465983d8deadfd642f96878791924cddd0687eafaa0fcf26
Jul 14 17:11:59.562582 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:11:59.562542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf" event={"ID":"18bee717-e3de-45f7-a7e2-da8e14181a46","Type":"ContainerStarted","Data":"f2de71f294ea4af8465983d8deadfd642f96878791924cddd0687eafaa0fcf26"}
Jul 14 17:12:01.883396 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:01.883365    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"]
Jul 14 17:12:01.888063 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:01.888038    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:01.890960 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:01.890934    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pk9bn\""
Jul 14 17:12:01.900598 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:01.900532    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"]
Jul 14 17:12:02.030432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:02.030397    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qs4n\" (UniqueName: \"kubernetes.io/projected/d8a58fa6-a994-4404-a50f-a91a6540543c-kube-api-access-7qs4n\") pod \"limitador-operator-controller-manager-859d7b78bd-bvbsr\" (UID: \"d8a58fa6-a994-4404-a50f-a91a6540543c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:02.131980 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:02.131896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qs4n\" (UniqueName: \"kubernetes.io/projected/d8a58fa6-a994-4404-a50f-a91a6540543c-kube-api-access-7qs4n\") pod \"limitador-operator-controller-manager-859d7b78bd-bvbsr\" (UID: \"d8a58fa6-a994-4404-a50f-a91a6540543c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:02.146447 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:02.146367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qs4n\" (UniqueName: \"kubernetes.io/projected/d8a58fa6-a994-4404-a50f-a91a6540543c-kube-api-access-7qs4n\") pod \"limitador-operator-controller-manager-859d7b78bd-bvbsr\" (UID: \"d8a58fa6-a994-4404-a50f-a91a6540543c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:02.205029 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:02.204991    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:02.349232 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:02.349202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"]
Jul 14 17:12:02.352077 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:12:02.352048    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a58fa6_a994_4404_a50f_a91a6540543c.slice/crio-1950a125b9052f3f2b6b52aad6ff6724e979bd621fc4ea6cd2fd0998d4fec198 WatchSource:0}: Error finding container 1950a125b9052f3f2b6b52aad6ff6724e979bd621fc4ea6cd2fd0998d4fec198: Status 404 returned error can't find the container with id 1950a125b9052f3f2b6b52aad6ff6724e979bd621fc4ea6cd2fd0998d4fec198
Jul 14 17:12:02.579427 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:02.579387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf" event={"ID":"18bee717-e3de-45f7-a7e2-da8e14181a46","Type":"ContainerStarted","Data":"e4f7c0537c3628dd58150ffc41b3d50e1ef8539e22f5598974eb1f9d3c1ae111"}
Jul 14 17:12:02.579632 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:02.579497    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf"
Jul 14 17:12:02.580678 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:02.580652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr" event={"ID":"d8a58fa6-a994-4404-a50f-a91a6540543c","Type":"ContainerStarted","Data":"1950a125b9052f3f2b6b52aad6ff6724e979bd621fc4ea6cd2fd0998d4fec198"}
Jul 14 17:12:02.597327 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:02.597282    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf" podStartSLOduration=1.838935182 podStartE2EDuration="4.597267283s" totalImagesPullingTime="2.758332101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:12:02.596528972 +0000 UTC m=+409.344128898" watchObservedRunningTime="2026-07-14 17:12:02.597267283 +0000 UTC m=+409.344867226"
Jul 14 17:12:05.420483 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.420445    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rmww5"]
Jul 14 17:12:05.425190 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.425154    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rmww5"
Jul 14 17:12:05.428228 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.428175    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-gz6cp\""
Jul 14 17:12:05.443514 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.443485    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rmww5"]
Jul 14 17:12:05.564974 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.564934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg5tc\" (UniqueName: \"kubernetes.io/projected/86c4237a-cc6e-4bbf-8806-3f5ad859a54f-kube-api-access-dg5tc\") pod \"authorino-operator-75cc8d5cb7-rmww5\" (UID: \"86c4237a-cc6e-4bbf-8806-3f5ad859a54f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rmww5"
Jul 14 17:12:05.595272 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.595238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf" event={"ID":"28fe39f8-a48d-4a4b-98cc-53012300aaef","Type":"ContainerStarted","Data":"fe2e5e9b8feecb45d026b420f2c42a2ab7e34d57a332fa58720da9aabb34ff9c"}
Jul 14 17:12:05.595426 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.595362    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:05.620592 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.620540    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf" podStartSLOduration=1.750880741 podStartE2EDuration="9.620520431s" totalImagesPullingTime="7.86963969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:12:05.618820856 +0000 UTC m=+412.366420778" watchObservedRunningTime="2026-07-14 17:12:05.620520431 +0000 UTC m=+412.368120307"
Jul 14 17:12:05.665564 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.665524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg5tc\" (UniqueName: \"kubernetes.io/projected/86c4237a-cc6e-4bbf-8806-3f5ad859a54f-kube-api-access-dg5tc\") pod \"authorino-operator-75cc8d5cb7-rmww5\" (UID: \"86c4237a-cc6e-4bbf-8806-3f5ad859a54f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rmww5"
Jul 14 17:12:05.675929 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.675758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg5tc\" (UniqueName: \"kubernetes.io/projected/86c4237a-cc6e-4bbf-8806-3f5ad859a54f-kube-api-access-dg5tc\") pod \"authorino-operator-75cc8d5cb7-rmww5\" (UID: \"86c4237a-cc6e-4bbf-8806-3f5ad859a54f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rmww5"
Jul 14 17:12:05.737552 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:05.737509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rmww5"
Jul 14 17:12:06.126632 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:06.126599    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rmww5"]
Jul 14 17:12:06.129641 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:12:06.129599    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c4237a_cc6e_4bbf_8806_3f5ad859a54f.slice/crio-81a91c60bea37657dd35c781c04761be840a386f6f19c536f0babdd643a7c71d WatchSource:0}: Error finding container 81a91c60bea37657dd35c781c04761be840a386f6f19c536f0babdd643a7c71d: Status 404 returned error can't find the container with id 81a91c60bea37657dd35c781c04761be840a386f6f19c536f0babdd643a7c71d
Jul 14 17:12:06.600476 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:06.600436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr" event={"ID":"d8a58fa6-a994-4404-a50f-a91a6540543c","Type":"ContainerStarted","Data":"5428ff809a969c9a823008e1e74ef6aa557478324cfb858f499b86b70af346f7"}
Jul 14 17:12:06.600922 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:06.600506    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:06.601712 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:06.601692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rmww5" event={"ID":"86c4237a-cc6e-4bbf-8806-3f5ad859a54f","Type":"ContainerStarted","Data":"81a91c60bea37657dd35c781c04761be840a386f6f19c536f0babdd643a7c71d"}
Jul 14 17:12:06.618784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:06.618739    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr" podStartSLOduration=1.959627022 podStartE2EDuration="5.61872265s" totalImagesPullingTime="3.659095628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:12:06.617672841 +0000 UTC m=+413.365272717" watchObservedRunningTime="2026-07-14 17:12:06.61872265 +0000 UTC m=+413.366322527"
Jul 14 17:12:07.662832 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.662795    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"]
Jul 14 17:12:07.667599 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.667567    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:07.674893 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.674868    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 14 17:12:07.681427 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.681393    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-25jch\""
Jul 14 17:12:07.681713 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.681696    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 14 17:12:07.685375 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.685350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b1ce680d-2369-4f20-b262-4b54934a0621-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fptzb\" (UID: \"b1ce680d-2369-4f20-b262-4b54934a0621\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:07.685621 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.685596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bthtr\" (UniqueName: \"kubernetes.io/projected/b1ce680d-2369-4f20-b262-4b54934a0621-kube-api-access-bthtr\") pod \"kuadrant-console-plugin-66b5774985-fptzb\" (UID: \"b1ce680d-2369-4f20-b262-4b54934a0621\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:07.685781 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.685763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1ce680d-2369-4f20-b262-4b54934a0621-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fptzb\" (UID: \"b1ce680d-2369-4f20-b262-4b54934a0621\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:07.686604 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.686581    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"]
Jul 14 17:12:07.786521 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.786483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1ce680d-2369-4f20-b262-4b54934a0621-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fptzb\" (UID: \"b1ce680d-2369-4f20-b262-4b54934a0621\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:07.786709 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.786575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b1ce680d-2369-4f20-b262-4b54934a0621-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fptzb\" (UID: \"b1ce680d-2369-4f20-b262-4b54934a0621\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:07.786709 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.786619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bthtr\" (UniqueName: \"kubernetes.io/projected/b1ce680d-2369-4f20-b262-4b54934a0621-kube-api-access-bthtr\") pod \"kuadrant-console-plugin-66b5774985-fptzb\" (UID: \"b1ce680d-2369-4f20-b262-4b54934a0621\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:07.787441 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.787412    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b1ce680d-2369-4f20-b262-4b54934a0621-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fptzb\" (UID: \"b1ce680d-2369-4f20-b262-4b54934a0621\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:07.789435 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.789407    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1ce680d-2369-4f20-b262-4b54934a0621-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fptzb\" (UID: \"b1ce680d-2369-4f20-b262-4b54934a0621\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:07.798243 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.798211    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bthtr\" (UniqueName: \"kubernetes.io/projected/b1ce680d-2369-4f20-b262-4b54934a0621-kube-api-access-bthtr\") pod \"kuadrant-console-plugin-66b5774985-fptzb\" (UID: \"b1ce680d-2369-4f20-b262-4b54934a0621\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:07.978319 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:07.978278    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"
Jul 14 17:12:08.401193 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:08.401158    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb"]
Jul 14 17:12:08.403098 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:12:08.403062    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1ce680d_2369_4f20_b262_4b54934a0621.slice/crio-04fcede1100c03060bba22b76b239a46bc4575de368e3dd5961376a53d063492 WatchSource:0}: Error finding container 04fcede1100c03060bba22b76b239a46bc4575de368e3dd5961376a53d063492: Status 404 returned error can't find the container with id 04fcede1100c03060bba22b76b239a46bc4575de368e3dd5961376a53d063492
Jul 14 17:12:08.612948 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:08.612865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb" event={"ID":"b1ce680d-2369-4f20-b262-4b54934a0621","Type":"ContainerStarted","Data":"04fcede1100c03060bba22b76b239a46bc4575de368e3dd5961376a53d063492"}
Jul 14 17:12:09.620115 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:09.620073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rmww5" event={"ID":"86c4237a-cc6e-4bbf-8806-3f5ad859a54f","Type":"ContainerStarted","Data":"c13fe6a89c7e5a5d8dc9b827248a90806687dd6c4c98288a082bd806356756cd"}
Jul 14 17:12:09.620549 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:09.620229    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rmww5"
Jul 14 17:12:09.641187 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:09.641123    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rmww5" podStartSLOduration=1.96334355 podStartE2EDuration="4.641103049s" totalImagesPullingTime="2.677759499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:12:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:12:09.640066338 +0000 UTC m=+416.387666238" watchObservedRunningTime="2026-07-14 17:12:09.641103049 +0000 UTC m=+416.388702997"
Jul 14 17:12:13.588014 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:13.587979    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sqltf"
Jul 14 17:12:16.604252 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:16.604224    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:17.609302 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:17.609272    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:18.623030 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.622999    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"]
Jul 14 17:12:18.623509 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.623220    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" containerName="manager" containerID="cri-o://fe2e5e9b8feecb45d026b420f2c42a2ab7e34d57a332fa58720da9aabb34ff9c" gracePeriod=2
Jul 14 17:12:18.633602 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.633575    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"]
Jul 14 17:12:18.642934 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.642903    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"]
Jul 14 17:12:18.643323 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.643129    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" containerName="manager" containerID="cri-o://5428ff809a969c9a823008e1e74ef6aa557478324cfb858f499b86b70af346f7" gracePeriod=2
Jul 14 17:12:18.660317 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.660244    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"]
Jul 14 17:12:18.664923 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.664881    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"]
Jul 14 17:12:18.665269 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:18.665233    2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d8a58fa6-a994-4404-a50f-a91a6540543c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-bvbsr\": pods \"limitador-operator-controller-manager-859d7b78bd-bvbsr\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:18.665361 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.665305    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" containerName="manager"
Jul 14 17:12:18.665361 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.665319    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" containerName="manager"
Jul 14 17:12:18.665361 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.665343    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" containerName="manager"
Jul 14 17:12:18.665361 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.665353    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" containerName="manager"
Jul 14 17:12:18.665483 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.665434    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" containerName="manager"
Jul 14 17:12:18.665483 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.665447    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" containerName="manager"
Jul 14 17:12:18.667526 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:18.667482    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:18.667830 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.667805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:18.672196 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:18.672157    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bvbsr\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:18.674676 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:18.674490    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:18.678701 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.678675    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt"]
Jul 14 17:12:18.680984 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.680962    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt"
Jul 14 17:12:18.683719 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.683614    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"]
Jul 14 17:12:18.696635 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.696559    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt"]
Jul 14 17:12:18.710111 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:18.710075    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bvbsr\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:18.712171 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:18.712136    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:18.782448 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.782406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqxmv\" (UniqueName: \"kubernetes.io/projected/4768c4fc-f8dd-4f8c-84cc-c2e4260706e5-kube-api-access-rqxmv\") pod \"limitador-operator-controller-manager-859d7b78bd-4vtpt\" (UID: \"4768c4fc-f8dd-4f8c-84cc-c2e4260706e5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt"
Jul 14 17:12:18.782720 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.782513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86jbs\" (UniqueName: \"kubernetes.io/projected/73dcaa9c-ba09-4471-9db8-54a3dae12d87-kube-api-access-86jbs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6j5wn\" (UID: \"73dcaa9c-ba09-4471-9db8-54a3dae12d87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:18.782720 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.782615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73dcaa9c-ba09-4471-9db8-54a3dae12d87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6j5wn\" (UID: \"73dcaa9c-ba09-4471-9db8-54a3dae12d87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:18.883661 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.883559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86jbs\" (UniqueName: \"kubernetes.io/projected/73dcaa9c-ba09-4471-9db8-54a3dae12d87-kube-api-access-86jbs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6j5wn\" (UID: \"73dcaa9c-ba09-4471-9db8-54a3dae12d87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:18.883661 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.883660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73dcaa9c-ba09-4471-9db8-54a3dae12d87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6j5wn\" (UID: \"73dcaa9c-ba09-4471-9db8-54a3dae12d87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:18.883911 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.883700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxmv\" (UniqueName: \"kubernetes.io/projected/4768c4fc-f8dd-4f8c-84cc-c2e4260706e5-kube-api-access-rqxmv\") pod \"limitador-operator-controller-manager-859d7b78bd-4vtpt\" (UID: \"4768c4fc-f8dd-4f8c-84cc-c2e4260706e5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt"
Jul 14 17:12:18.884160 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.884132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73dcaa9c-ba09-4471-9db8-54a3dae12d87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6j5wn\" (UID: \"73dcaa9c-ba09-4471-9db8-54a3dae12d87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:18.896546 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.896518    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86jbs\" (UniqueName: \"kubernetes.io/projected/73dcaa9c-ba09-4471-9db8-54a3dae12d87-kube-api-access-86jbs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6j5wn\" (UID: \"73dcaa9c-ba09-4471-9db8-54a3dae12d87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:18.896732 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:18.896709    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqxmv\" (UniqueName: \"kubernetes.io/projected/4768c4fc-f8dd-4f8c-84cc-c2e4260706e5-kube-api-access-rqxmv\") pod \"limitador-operator-controller-manager-859d7b78bd-4vtpt\" (UID: \"4768c4fc-f8dd-4f8c-84cc-c2e4260706e5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt"
Jul 14 17:12:19.024391 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:19.024347    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:19.033346 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:19.033311    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt"
Jul 14 17:12:20.628279 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:20.628235    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rmww5"
Jul 14 17:12:32.748966 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.748813    2580 generic.go:356] "Generic (PLEG): container finished" podID="d8a58fa6-a994-4404-a50f-a91a6540543c" containerID="5428ff809a969c9a823008e1e74ef6aa557478324cfb858f499b86b70af346f7" exitCode=0
Jul 14 17:12:32.752702 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.752567    2580 generic.go:356] "Generic (PLEG): container finished" podID="28fe39f8-a48d-4a4b-98cc-53012300aaef" containerID="fe2e5e9b8feecb45d026b420f2c42a2ab7e34d57a332fa58720da9aabb34ff9c" exitCode=0
Jul 14 17:12:32.818596 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.818571    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:32.820740 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:32.820704    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:32.821918 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.821897    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:32.824029 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:32.823995    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bvbsr\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:32.827742 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:32.827695    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:32.830060 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.830035    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"]
Jul 14 17:12:32.830654 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:12:32.830622    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73dcaa9c_ba09_4471_9db8_54a3dae12d87.slice/crio-e8022c465f9cd2296ebaec408d631d59a3c3502c1fe1297e83b32df082a110e0 WatchSource:0}: Error finding container e8022c465f9cd2296ebaec408d631d59a3c3502c1fe1297e83b32df082a110e0: Status 404 returned error can't find the container with id e8022c465f9cd2296ebaec408d631d59a3c3502c1fe1297e83b32df082a110e0
Jul 14 17:12:32.846644 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.846621    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt"]
Jul 14 17:12:32.849173 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:12:32.849132    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4768c4fc_f8dd_4f8c_84cc_c2e4260706e5.slice/crio-4b999edd5c348ceb670763b6ebf337a43eada51986426adc976393fb5cde301e WatchSource:0}: Error finding container 4b999edd5c348ceb670763b6ebf337a43eada51986426adc976393fb5cde301e: Status 404 returned error can't find the container with id 4b999edd5c348ceb670763b6ebf337a43eada51986426adc976393fb5cde301e
Jul 14 17:12:32.916161 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.916130    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6484j\" (UniqueName: \"kubernetes.io/projected/28fe39f8-a48d-4a4b-98cc-53012300aaef-kube-api-access-6484j\") pod \"28fe39f8-a48d-4a4b-98cc-53012300aaef\" (UID: \"28fe39f8-a48d-4a4b-98cc-53012300aaef\") "
Jul 14 17:12:32.916161 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.916163    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qs4n\" (UniqueName: \"kubernetes.io/projected/d8a58fa6-a994-4404-a50f-a91a6540543c-kube-api-access-7qs4n\") pod \"d8a58fa6-a994-4404-a50f-a91a6540543c\" (UID: \"d8a58fa6-a994-4404-a50f-a91a6540543c\") "
Jul 14 17:12:32.916400 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.916234    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28fe39f8-a48d-4a4b-98cc-53012300aaef-extensions-socket-volume\") pod \"28fe39f8-a48d-4a4b-98cc-53012300aaef\" (UID: \"28fe39f8-a48d-4a4b-98cc-53012300aaef\") "
Jul 14 17:12:32.916624 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.916601    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28fe39f8-a48d-4a4b-98cc-53012300aaef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "28fe39f8-a48d-4a4b-98cc-53012300aaef" (UID: "28fe39f8-a48d-4a4b-98cc-53012300aaef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:12:32.967414 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.967360    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28fe39f8-a48d-4a4b-98cc-53012300aaef-kube-api-access-6484j" (OuterVolumeSpecName: "kube-api-access-6484j") pod "28fe39f8-a48d-4a4b-98cc-53012300aaef" (UID: "28fe39f8-a48d-4a4b-98cc-53012300aaef"). InnerVolumeSpecName "kube-api-access-6484j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:12:32.967414 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:32.967364    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8a58fa6-a994-4404-a50f-a91a6540543c-kube-api-access-7qs4n" (OuterVolumeSpecName: "kube-api-access-7qs4n") pod "d8a58fa6-a994-4404-a50f-a91a6540543c" (UID: "d8a58fa6-a994-4404-a50f-a91a6540543c"). InnerVolumeSpecName "kube-api-access-7qs4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:12:33.017008 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.016964    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6484j\" (UniqueName: \"kubernetes.io/projected/28fe39f8-a48d-4a4b-98cc-53012300aaef-kube-api-access-6484j\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:12:33.017008 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.017000    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qs4n\" (UniqueName: \"kubernetes.io/projected/d8a58fa6-a994-4404-a50f-a91a6540543c-kube-api-access-7qs4n\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:12:33.017008 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.017015    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28fe39f8-a48d-4a4b-98cc-53012300aaef-extensions-socket-volume\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:12:33.763652 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.763601    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn" event={"ID":"73dcaa9c-ba09-4471-9db8-54a3dae12d87","Type":"ContainerStarted","Data":"70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925"}
Jul 14 17:12:33.764145 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.763702    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:33.764145 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.763728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn" event={"ID":"73dcaa9c-ba09-4471-9db8-54a3dae12d87","Type":"ContainerStarted","Data":"e8022c465f9cd2296ebaec408d631d59a3c3502c1fe1297e83b32df082a110e0"}
Jul 14 17:12:33.765042 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.765016    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:33.765042 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.765020    2580 scope.go:117] "RemoveContainer" containerID="5428ff809a969c9a823008e1e74ef6aa557478324cfb858f499b86b70af346f7"
Jul 14 17:12:33.766371 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:33.766171    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bvbsr\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:33.766516 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.766495    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:33.768328 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:33.768294    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:33.768610 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.768577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt" event={"ID":"4768c4fc-f8dd-4f8c-84cc-c2e4260706e5","Type":"ContainerStarted","Data":"89069ac818a5c2a1ea89eab265cacc2e303a7f094926b8014666f86dd29aa6c1"}
Jul 14 17:12:33.768727 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.768646    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt"
Jul 14 17:12:33.768727 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.768672    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt" event={"ID":"4768c4fc-f8dd-4f8c-84cc-c2e4260706e5","Type":"ContainerStarted","Data":"4b999edd5c348ceb670763b6ebf337a43eada51986426adc976393fb5cde301e"}
Jul 14 17:12:33.770621 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.770596    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb" event={"ID":"b1ce680d-2369-4f20-b262-4b54934a0621","Type":"ContainerStarted","Data":"f6e26dcdd212a163a1e113b8a9ce1d83bbe5218cff9c4cc3d5b4ccc47e987147"}
Jul 14 17:12:33.780393 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.780371    2580 scope.go:117] "RemoveContainer" containerID="fe2e5e9b8feecb45d026b420f2c42a2ab7e34d57a332fa58720da9aabb34ff9c"
Jul 14 17:12:33.795445 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.795399    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn" podStartSLOduration=15.795385174 podStartE2EDuration="15.795385174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:12:33.793988453 +0000 UTC m=+440.541588350" watchObservedRunningTime="2026-07-14 17:12:33.795385174 +0000 UTC m=+440.542985050"
Jul 14 17:12:33.796085 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:33.796058    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:33.816280 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.816232    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fptzb" podStartSLOduration=2.475248966 podStartE2EDuration="26.816215238s" totalImagesPullingTime="24.340966272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:12:33.81423292 +0000 UTC m=+440.561832791" watchObservedRunningTime="2026-07-14 17:12:33.816215238 +0000 UTC m=+440.563815114"
Jul 14 17:12:33.836904 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.836834    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt" podStartSLOduration=15.836812311 podStartE2EDuration="15.836812311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:12:33.836528283 +0000 UTC m=+440.584128181" watchObservedRunningTime="2026-07-14 17:12:33.836812311 +0000 UTC m=+440.584412204"
Jul 14 17:12:33.838583 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:33.838558    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bvbsr\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:33.879810 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:33.879770    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bvbsr\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvbsr"
Jul 14 17:12:33.881559 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.881534    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" path="/var/lib/kubelet/pods/28fe39f8-a48d-4a4b-98cc-53012300aaef/volumes"
Jul 14 17:12:33.881885 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:33.881870    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8a58fa6-a994-4404-a50f-a91a6540543c" path="/var/lib/kubelet/pods/d8a58fa6-a994-4404-a50f-a91a6540543c/volumes"
Jul 14 17:12:33.881944 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:33.881908    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qftdf\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="28fe39f8-a48d-4a4b-98cc-53012300aaef" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qftdf"
Jul 14 17:12:36.803036 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:36.802993    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"]
Jul 14 17:12:36.803425 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:36.803185    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn" podUID="73dcaa9c-ba09-4471-9db8-54a3dae12d87" containerName="manager" containerID="cri-o://70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925" gracePeriod=10
Jul 14 17:12:37.049304 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.049280    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:37.154452 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.154360    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73dcaa9c-ba09-4471-9db8-54a3dae12d87-extensions-socket-volume\") pod \"73dcaa9c-ba09-4471-9db8-54a3dae12d87\" (UID: \"73dcaa9c-ba09-4471-9db8-54a3dae12d87\") "
Jul 14 17:12:37.154452 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.154427    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86jbs\" (UniqueName: \"kubernetes.io/projected/73dcaa9c-ba09-4471-9db8-54a3dae12d87-kube-api-access-86jbs\") pod \"73dcaa9c-ba09-4471-9db8-54a3dae12d87\" (UID: \"73dcaa9c-ba09-4471-9db8-54a3dae12d87\") "
Jul 14 17:12:37.154748 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.154720    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73dcaa9c-ba09-4471-9db8-54a3dae12d87-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "73dcaa9c-ba09-4471-9db8-54a3dae12d87" (UID: "73dcaa9c-ba09-4471-9db8-54a3dae12d87"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:12:37.156700 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.156669    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73dcaa9c-ba09-4471-9db8-54a3dae12d87-kube-api-access-86jbs" (OuterVolumeSpecName: "kube-api-access-86jbs") pod "73dcaa9c-ba09-4471-9db8-54a3dae12d87" (UID: "73dcaa9c-ba09-4471-9db8-54a3dae12d87"). InnerVolumeSpecName "kube-api-access-86jbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:12:37.255789 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.255752    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73dcaa9c-ba09-4471-9db8-54a3dae12d87-extensions-socket-volume\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:12:37.255789 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.255779    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86jbs\" (UniqueName: \"kubernetes.io/projected/73dcaa9c-ba09-4471-9db8-54a3dae12d87-kube-api-access-86jbs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:12:37.791139 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.791103    2580 generic.go:356] "Generic (PLEG): container finished" podID="73dcaa9c-ba09-4471-9db8-54a3dae12d87" containerID="70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925" exitCode=0
Jul 14 17:12:37.791335 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.791161    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"
Jul 14 17:12:37.791335 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.791162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn" event={"ID":"73dcaa9c-ba09-4471-9db8-54a3dae12d87","Type":"ContainerDied","Data":"70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925"}
Jul 14 17:12:37.791335 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.791281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn" event={"ID":"73dcaa9c-ba09-4471-9db8-54a3dae12d87","Type":"ContainerDied","Data":"e8022c465f9cd2296ebaec408d631d59a3c3502c1fe1297e83b32df082a110e0"}
Jul 14 17:12:37.791335 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.791302    2580 scope.go:117] "RemoveContainer" containerID="70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925"
Jul 14 17:12:37.800230 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.800213    2580 scope.go:117] "RemoveContainer" containerID="70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925"
Jul 14 17:12:37.800503 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:12:37.800483    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925\": container with ID starting with 70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925 not found: ID does not exist" containerID="70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925"
Jul 14 17:12:37.800583 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.800511    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925"} err="failed to get container status \"70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925\": rpc error: code = NotFound desc = could not find container \"70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925\": container with ID starting with 70db94248f90bc6217158b4ce59fd6b4bdcee69f93a7d8310c985e1ede03e925 not found: ID does not exist"
Jul 14 17:12:37.830670 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.827149    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"]
Jul 14 17:12:37.832191 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.832169    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6j5wn"]
Jul 14 17:12:37.882746 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:37.882714    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73dcaa9c-ba09-4471-9db8-54a3dae12d87" path="/var/lib/kubelet/pods/73dcaa9c-ba09-4471-9db8-54a3dae12d87/volumes"
Jul 14 17:12:44.780120 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:44.780088    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vtpt"
Jul 14 17:12:53.146907 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.146865    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"]
Jul 14 17:12:53.147263 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.147238    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73dcaa9c-ba09-4471-9db8-54a3dae12d87" containerName="manager"
Jul 14 17:12:53.147263 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.147250    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="73dcaa9c-ba09-4471-9db8-54a3dae12d87" containerName="manager"
Jul 14 17:12:53.147333 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.147327    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="73dcaa9c-ba09-4471-9db8-54a3dae12d87" containerName="manager"
Jul 14 17:12:53.317642 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.317604    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"]
Jul 14 17:12:53.317801 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.317731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.320981 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.320963    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 14 17:12:53.321043 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.320966    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bm7xm\""
Jul 14 17:12:53.385685 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.385650    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.385897 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.385699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.385897 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.385728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.385897 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.385788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.385897 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.385833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.386062 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.385898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.386062 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.385927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.386062 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.385960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6b92\" (UniqueName: \"kubernetes.io/projected/80fadef6-b849-463d-9ead-35c0ca1bc8cd-kube-api-access-p6b92\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.386062 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.386014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486423 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486423 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6b92\" (UniqueName: \"kubernetes.io/projected/80fadef6-b849-463d-9ead-35c0ca1bc8cd-kube-api-access-p6b92\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486642 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486642 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486745 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486745 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486745 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486931 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486931 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486931 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.486931 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.486907    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.487135 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.487077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.487207 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.487160    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.487438 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.487371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.489224 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.489198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.489332 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.489308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.495601 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.495574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6b92\" (UniqueName: \"kubernetes.io/projected/80fadef6-b849-463d-9ead-35c0ca1bc8cd-kube-api-access-p6b92\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.496330 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.496304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80fadef6-b849-463d-9ead-35c0ca1bc8cd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-nd7hw\" (UID: \"80fadef6-b849-463d-9ead-35c0ca1bc8cd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.628291 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.628260    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:53.799306 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.799263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"]
Jul 14 17:12:53.801919 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:12:53.801891    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80fadef6_b849_463d_9ead_35c0ca1bc8cd.slice/crio-f8ff4a6baa2b8f211ad6b5279c661d288d577b14850feee1f68a65439e7657b9 WatchSource:0}: Error finding container f8ff4a6baa2b8f211ad6b5279c661d288d577b14850feee1f68a65439e7657b9: Status 404 returned error can't find the container with id f8ff4a6baa2b8f211ad6b5279c661d288d577b14850feee1f68a65439e7657b9
Jul 14 17:12:53.864214 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:53.864175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw" event={"ID":"80fadef6-b849-463d-9ead-35c0ca1bc8cd","Type":"ContainerStarted","Data":"f8ff4a6baa2b8f211ad6b5279c661d288d577b14850feee1f68a65439e7657b9"}
Jul 14 17:12:56.974442 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:56.974405    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:12:56.974705 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:56.974480    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:12:56.974705 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:56.974508    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:12:57.885369 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:57.885337    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw" event={"ID":"80fadef6-b849-463d-9ead-35c0ca1bc8cd","Type":"ContainerStarted","Data":"7e782c6254445f775abea5a5cf6fe0b3d69c389298db134b9f4bc1576463e94c"}
Jul 14 17:12:57.911302 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:57.911251    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw" podStartSLOduration=1.741217544 podStartE2EDuration="4.91123358s" totalImagesPullingTime="3.170016036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:12:57.909227407 +0000 UTC m=+464.656827296" watchObservedRunningTime="2026-07-14 17:12:57.91123358 +0000 UTC m=+464.658833457"
Jul 14 17:12:58.629440 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:58.629401    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:58.635038 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:58.635009    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:58.890159 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:58.890078    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:58.891087 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:58.891068    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:12:58.974721 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:58.974682    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"]
Jul 14 17:12:59.014035 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.014000    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"]
Jul 14 17:12:59.014203 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.014115    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:12:59.016811 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.016790    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 14 17:12:59.028161 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.028137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk8mj\" (UniqueName: \"kubernetes.io/projected/7157fed9-bde2-4d81-b775-63d23713ee04-kube-api-access-hk8mj\") pod \"limitador-limitador-64c94fd64f-xhqsh\" (UID: \"7157fed9-bde2-4d81-b775-63d23713ee04\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:12:59.028288 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.028186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7157fed9-bde2-4d81-b775-63d23713ee04-config-file\") pod \"limitador-limitador-64c94fd64f-xhqsh\" (UID: \"7157fed9-bde2-4d81-b775-63d23713ee04\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:12:59.072816 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.072785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"]
Jul 14 17:12:59.128996 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.128958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7157fed9-bde2-4d81-b775-63d23713ee04-config-file\") pod \"limitador-limitador-64c94fd64f-xhqsh\" (UID: \"7157fed9-bde2-4d81-b775-63d23713ee04\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:12:59.129190 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.129069    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk8mj\" (UniqueName: \"kubernetes.io/projected/7157fed9-bde2-4d81-b775-63d23713ee04-kube-api-access-hk8mj\") pod \"limitador-limitador-64c94fd64f-xhqsh\" (UID: \"7157fed9-bde2-4d81-b775-63d23713ee04\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:12:59.129667 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.129646    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7157fed9-bde2-4d81-b775-63d23713ee04-config-file\") pod \"limitador-limitador-64c94fd64f-xhqsh\" (UID: \"7157fed9-bde2-4d81-b775-63d23713ee04\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:12:59.141995 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.141909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk8mj\" (UniqueName: \"kubernetes.io/projected/7157fed9-bde2-4d81-b775-63d23713ee04-kube-api-access-hk8mj\") pod \"limitador-limitador-64c94fd64f-xhqsh\" (UID: \"7157fed9-bde2-4d81-b775-63d23713ee04\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:12:59.325076 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.325045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:12:59.408652 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.408566    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:12:59.475959 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.475928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:12:59.476087 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.475974    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:12:59.476087 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.476064    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw"
Jul 14 17:12:59.531817 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.531783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f-config-file\") pod \"limitador-limitador-54d6d988d5-77fzw\" (UID: \"8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw"
Jul 14 17:12:59.532548 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.532516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsn9z\" (UniqueName: \"kubernetes.io/projected/8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f-kube-api-access-xsn9z\") pod \"limitador-limitador-54d6d988d5-77fzw\" (UID: \"8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw"
Jul 14 17:12:59.608486 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.608447    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"]
Jul 14 17:12:59.619199 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:12:59.619165    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7157fed9_bde2_4d81_b775_63d23713ee04.slice/crio-8e1d71f83cdf34b14c637ec645c22e8461a70f72c19f1b90d8cbb5d4a67885d0 WatchSource:0}: Error finding container 8e1d71f83cdf34b14c637ec645c22e8461a70f72c19f1b90d8cbb5d4a67885d0: Status 404 returned error can't find the container with id 8e1d71f83cdf34b14c637ec645c22e8461a70f72c19f1b90d8cbb5d4a67885d0
Jul 14 17:12:59.633794 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.633770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f-config-file\") pod \"limitador-limitador-54d6d988d5-77fzw\" (UID: \"8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw"
Jul 14 17:12:59.634135 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.633833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsn9z\" (UniqueName: \"kubernetes.io/projected/8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f-kube-api-access-xsn9z\") pod \"limitador-limitador-54d6d988d5-77fzw\" (UID: \"8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw"
Jul 14 17:12:59.634376 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.634355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f-config-file\") pod \"limitador-limitador-54d6d988d5-77fzw\" (UID: \"8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw"
Jul 14 17:12:59.644226 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.644199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsn9z\" (UniqueName: \"kubernetes.io/projected/8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f-kube-api-access-xsn9z\") pod \"limitador-limitador-54d6d988d5-77fzw\" (UID: \"8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw"
Jul 14 17:12:59.797251 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.797209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw"
Jul 14 17:12:59.896292 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:12:59.896261    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh" event={"ID":"7157fed9-bde2-4d81-b775-63d23713ee04","Type":"ContainerStarted","Data":"8e1d71f83cdf34b14c637ec645c22e8461a70f72c19f1b90d8cbb5d4a67885d0"}
Jul 14 17:13:00.056693 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:00.056607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:13:00.059933 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:13:00.059897    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f4af9a4_0d4c_4fd4_bf99_23c7a5fa887f.slice/crio-b3b6c6453243a654c36c28b75cbf316c8ed894e0913baa068e11f0894b665c26 WatchSource:0}: Error finding container b3b6c6453243a654c36c28b75cbf316c8ed894e0913baa068e11f0894b665c26: Status 404 returned error can't find the container with id b3b6c6453243a654c36c28b75cbf316c8ed894e0913baa068e11f0894b665c26
Jul 14 17:13:00.902370 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:00.902335    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw" event={"ID":"8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f","Type":"ContainerStarted","Data":"b3b6c6453243a654c36c28b75cbf316c8ed894e0913baa068e11f0894b665c26"}
Jul 14 17:13:03.921239 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:03.921199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh" event={"ID":"7157fed9-bde2-4d81-b775-63d23713ee04","Type":"ContainerStarted","Data":"e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd"}
Jul 14 17:13:03.921676 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:03.921331    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:13:03.922780 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:03.922755    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw" event={"ID":"8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f","Type":"ContainerStarted","Data":"a2f571a3181a0389aecd80ad7d7db74d20a8ff5825e45b820721018242253538"}
Jul 14 17:13:03.922904 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:03.922889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw"
Jul 14 17:13:03.939719 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:03.939639    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh" podStartSLOduration=1.8710520910000001 podStartE2EDuration="5.939621268s" totalImagesPullingTime="4.068569177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:13:03.939521866 +0000 UTC m=+470.687121764" watchObservedRunningTime="2026-07-14 17:13:03.939621268 +0000 UTC m=+470.687221148"
Jul 14 17:13:03.958930 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:03.958867    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw" podStartSLOduration=1.324847703 podStartE2EDuration="4.958823817s" totalImagesPullingTime="3.633976114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:13:03.958017409 +0000 UTC m=+470.705617307" watchObservedRunningTime="2026-07-14 17:13:03.958823817 +0000 UTC m=+470.706423694"
Jul 14 17:13:14.930987 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:14.930953    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-77fzw"
Jul 14 17:13:14.931397 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:14.931044    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:13:14.997338 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:14.997306    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"]
Jul 14 17:13:14.997498 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:14.997474    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh" podUID="7157fed9-bde2-4d81-b775-63d23713ee04" containerName="limitador" containerID="cri-o://e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd" gracePeriod=30
Jul 14 17:13:15.544350 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.544326    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:13:15.667108 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.667018    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7157fed9-bde2-4d81-b775-63d23713ee04-config-file\") pod \"7157fed9-bde2-4d81-b775-63d23713ee04\" (UID: \"7157fed9-bde2-4d81-b775-63d23713ee04\") "
Jul 14 17:13:15.667108 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.667075    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk8mj\" (UniqueName: \"kubernetes.io/projected/7157fed9-bde2-4d81-b775-63d23713ee04-kube-api-access-hk8mj\") pod \"7157fed9-bde2-4d81-b775-63d23713ee04\" (UID: \"7157fed9-bde2-4d81-b775-63d23713ee04\") "
Jul 14 17:13:15.667447 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.667425    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7157fed9-bde2-4d81-b775-63d23713ee04-config-file" (OuterVolumeSpecName: "config-file") pod "7157fed9-bde2-4d81-b775-63d23713ee04" (UID: "7157fed9-bde2-4d81-b775-63d23713ee04"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:13:15.669391 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.669363    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7157fed9-bde2-4d81-b775-63d23713ee04-kube-api-access-hk8mj" (OuterVolumeSpecName: "kube-api-access-hk8mj") pod "7157fed9-bde2-4d81-b775-63d23713ee04" (UID: "7157fed9-bde2-4d81-b775-63d23713ee04"). InnerVolumeSpecName "kube-api-access-hk8mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:13:15.767699 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.767663    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7157fed9-bde2-4d81-b775-63d23713ee04-config-file\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:13:15.767699 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.767696    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk8mj\" (UniqueName: \"kubernetes.io/projected/7157fed9-bde2-4d81-b775-63d23713ee04-kube-api-access-hk8mj\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:13:15.981972 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.981940    2580 generic.go:356] "Generic (PLEG): container finished" podID="7157fed9-bde2-4d81-b775-63d23713ee04" containerID="e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd" exitCode=0
Jul 14 17:13:15.982361 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.982001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh" event={"ID":"7157fed9-bde2-4d81-b775-63d23713ee04","Type":"ContainerDied","Data":"e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd"}
Jul 14 17:13:15.982361 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.982012    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"
Jul 14 17:13:15.982361 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.982043    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xhqsh" event={"ID":"7157fed9-bde2-4d81-b775-63d23713ee04","Type":"ContainerDied","Data":"8e1d71f83cdf34b14c637ec645c22e8461a70f72c19f1b90d8cbb5d4a67885d0"}
Jul 14 17:13:15.982361 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.982062    2580 scope.go:117] "RemoveContainer" containerID="e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd"
Jul 14 17:13:15.990488 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.990469    2580 scope.go:117] "RemoveContainer" containerID="e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd"
Jul 14 17:13:15.990755 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:13:15.990735    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd\": container with ID starting with e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd not found: ID does not exist" containerID="e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd"
Jul 14 17:13:15.990819 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:15.990769    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd"} err="failed to get container status \"e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd\": rpc error: code = NotFound desc = could not find container \"e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd\": container with ID starting with e401aeecd85ce498a20e65be90658aac56dc5b052aa42c634e6e04438dd709bd not found: ID does not exist"
Jul 14 17:13:16.000374 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:16.000346    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"]
Jul 14 17:13:16.007452 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:16.007429    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xhqsh"]
Jul 14 17:13:17.882081 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:17.882045    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7157fed9-bde2-4d81-b775-63d23713ee04" path="/var/lib/kubelet/pods/7157fed9-bde2-4d81-b775-63d23713ee04/volumes"
Jul 14 17:13:41.170952 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.170911    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64846ff95c-b7964"]
Jul 14 17:13:41.171810 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.171396    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7157fed9-bde2-4d81-b775-63d23713ee04" containerName="limitador"
Jul 14 17:13:41.171810 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.171414    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7157fed9-bde2-4d81-b775-63d23713ee04" containerName="limitador"
Jul 14 17:13:41.171810 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.171513    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7157fed9-bde2-4d81-b775-63d23713ee04" containerName="limitador"
Jul 14 17:13:41.177890 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.177866    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:13:41.180924 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.180895    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 14 17:13:41.180924 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.180915    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 14 17:13:41.181103 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.181055    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-r25cv\""
Jul 14 17:13:41.204086 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.204051    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64846ff95c-b7964"]
Jul 14 17:13:41.291726 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.291694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgzrd\" (UniqueName: \"kubernetes.io/projected/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-kube-api-access-qgzrd\") pod \"maas-controller-64846ff95c-b7964\" (UID: \"75b9ec7c-b76d-4266-ae5f-576cf90aff9f\") " pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:13:41.291726 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.291732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-webhook-cert\") pod \"maas-controller-64846ff95c-b7964\" (UID: \"75b9ec7c-b76d-4266-ae5f-576cf90aff9f\") " pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:13:41.393132 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.393090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgzrd\" (UniqueName: \"kubernetes.io/projected/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-kube-api-access-qgzrd\") pod \"maas-controller-64846ff95c-b7964\" (UID: \"75b9ec7c-b76d-4266-ae5f-576cf90aff9f\") " pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:13:41.393132 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.393142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-webhook-cert\") pod \"maas-controller-64846ff95c-b7964\" (UID: \"75b9ec7c-b76d-4266-ae5f-576cf90aff9f\") " pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:13:41.395737 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.395715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-webhook-cert\") pod \"maas-controller-64846ff95c-b7964\" (UID: \"75b9ec7c-b76d-4266-ae5f-576cf90aff9f\") " pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:13:41.404832 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.404803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgzrd\" (UniqueName: \"kubernetes.io/projected/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-kube-api-access-qgzrd\") pod \"maas-controller-64846ff95c-b7964\" (UID: \"75b9ec7c-b76d-4266-ae5f-576cf90aff9f\") " pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:13:41.487956 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.487921    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:13:41.631355 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:41.631264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64846ff95c-b7964"]
Jul 14 17:13:41.633639 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:13:41.633607    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b9ec7c_b76d_4266_ae5f_576cf90aff9f.slice/crio-aa860fc4c21f95fffe1927785bce0c153702d415ec27f0037ce88691834c7c74 WatchSource:0}: Error finding container aa860fc4c21f95fffe1927785bce0c153702d415ec27f0037ce88691834c7c74: Status 404 returned error can't find the container with id aa860fc4c21f95fffe1927785bce0c153702d415ec27f0037ce88691834c7c74
Jul 14 17:13:42.102651 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:42.102619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64846ff95c-b7964" event={"ID":"75b9ec7c-b76d-4266-ae5f-576cf90aff9f","Type":"ContainerStarted","Data":"aa860fc4c21f95fffe1927785bce0c153702d415ec27f0037ce88691834c7c74"}
Jul 14 17:13:44.114707 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:44.114673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64846ff95c-b7964" event={"ID":"75b9ec7c-b76d-4266-ae5f-576cf90aff9f","Type":"ContainerStarted","Data":"a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867"}
Jul 14 17:13:44.115132 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:44.114757    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:13:44.156889 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:44.156814    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64846ff95c-b7964" podStartSLOduration=1.281029323 podStartE2EDuration="3.15679863s" totalImagesPullingTime="1.875769307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:13:44.15327909 +0000 UTC m=+510.900878985" watchObservedRunningTime="2026-07-14 17:13:44.15679863 +0000 UTC m=+510.904398507"
Jul 14 17:13:55.121362 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:13:55.121318    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:14:16.003329 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.003290    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54675d56c-659bs"]
Jul 14 17:14:16.007743 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.007721    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.010537 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.010514    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5zbzj\""
Jul 14 17:14:16.010662 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.010515    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 17:14:16.011420 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.011407    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 14 17:14:16.011469 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.011431    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 17:14:16.028752 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.028716    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54675d56c-659bs"]
Jul 14 17:14:16.077047 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.077012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts9kf\" (UniqueName: \"kubernetes.io/projected/ed7e0b20-692f-43a1-9e55-4aa569e4a031-kube-api-access-ts9kf\") pod \"maas-api-54675d56c-659bs\" (UID: \"ed7e0b20-692f-43a1-9e55-4aa569e4a031\") " pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.077224 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.077085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed7e0b20-692f-43a1-9e55-4aa569e4a031-openshift-service-ca\") pod \"maas-api-54675d56c-659bs\" (UID: \"ed7e0b20-692f-43a1-9e55-4aa569e4a031\") " pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.077224 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.077120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed7e0b20-692f-43a1-9e55-4aa569e4a031-maas-api-tls\") pod \"maas-api-54675d56c-659bs\" (UID: \"ed7e0b20-692f-43a1-9e55-4aa569e4a031\") " pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.178032 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.177997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed7e0b20-692f-43a1-9e55-4aa569e4a031-openshift-service-ca\") pod \"maas-api-54675d56c-659bs\" (UID: \"ed7e0b20-692f-43a1-9e55-4aa569e4a031\") " pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.178230 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.178056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed7e0b20-692f-43a1-9e55-4aa569e4a031-maas-api-tls\") pod \"maas-api-54675d56c-659bs\" (UID: \"ed7e0b20-692f-43a1-9e55-4aa569e4a031\") " pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.178230 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.178129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts9kf\" (UniqueName: \"kubernetes.io/projected/ed7e0b20-692f-43a1-9e55-4aa569e4a031-kube-api-access-ts9kf\") pod \"maas-api-54675d56c-659bs\" (UID: \"ed7e0b20-692f-43a1-9e55-4aa569e4a031\") " pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.178655 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.178626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed7e0b20-692f-43a1-9e55-4aa569e4a031-openshift-service-ca\") pod \"maas-api-54675d56c-659bs\" (UID: \"ed7e0b20-692f-43a1-9e55-4aa569e4a031\") " pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.180595 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.180572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed7e0b20-692f-43a1-9e55-4aa569e4a031-maas-api-tls\") pod \"maas-api-54675d56c-659bs\" (UID: \"ed7e0b20-692f-43a1-9e55-4aa569e4a031\") " pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.189485 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.189463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts9kf\" (UniqueName: \"kubernetes.io/projected/ed7e0b20-692f-43a1-9e55-4aa569e4a031-kube-api-access-ts9kf\") pod \"maas-api-54675d56c-659bs\" (UID: \"ed7e0b20-692f-43a1-9e55-4aa569e4a031\") " pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.318420 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.318333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:16.468463 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.468418    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54675d56c-659bs"]
Jul 14 17:14:16.473362 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:14:16.473335    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7e0b20_692f_43a1_9e55_4aa569e4a031.slice/crio-09ea3f716253829d894dd1571edc4d65d035934d872c54116c2d9447e0fa9665 WatchSource:0}: Error finding container 09ea3f716253829d894dd1571edc4d65d035934d872c54116c2d9447e0fa9665: Status 404 returned error can't find the container with id 09ea3f716253829d894dd1571edc4d65d035934d872c54116c2d9447e0fa9665
Jul 14 17:14:16.530835 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.530801    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"]
Jul 14 17:14:16.536872 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.536838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"
Jul 14 17:14:16.539533 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.539514    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 14 17:14:16.543187 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.543139    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"]
Jul 14 17:14:16.552027 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.552000    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"]
Jul 14 17:14:16.556169 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.556142    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"
Jul 14 17:14:16.558737 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.558711    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-848tw\""
Jul 14 17:14:16.567901 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.566451    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"]
Jul 14 17:14:16.582128 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.582058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baac413f-f096-4cf1-98ba-bd5b8a8dc8f7-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-z6rq6\" (UID: \"baac413f-f096-4cf1-98ba-bd5b8a8dc8f7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"
Jul 14 17:14:16.582128 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.582101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v865c\" (UniqueName: \"kubernetes.io/projected/7560cab2-d533-4a56-8a9d-497347e810de-kube-api-access-v865c\") pod \"payload-pre-processing-74584bb9f8-zsl6p\" (UID: \"7560cab2-d533-4a56-8a9d-497347e810de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"
Jul 14 17:14:16.582291 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.582132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7560cab2-d533-4a56-8a9d-497347e810de-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-zsl6p\" (UID: \"7560cab2-d533-4a56-8a9d-497347e810de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"
Jul 14 17:14:16.582291 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.582174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpggl\" (UniqueName: \"kubernetes.io/projected/baac413f-f096-4cf1-98ba-bd5b8a8dc8f7-kube-api-access-cpggl\") pod \"payload-processing-f9b8d9b9b-z6rq6\" (UID: \"baac413f-f096-4cf1-98ba-bd5b8a8dc8f7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"
Jul 14 17:14:16.682821 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.682783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7560cab2-d533-4a56-8a9d-497347e810de-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-zsl6p\" (UID: \"7560cab2-d533-4a56-8a9d-497347e810de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"
Jul 14 17:14:16.683025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.682831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpggl\" (UniqueName: \"kubernetes.io/projected/baac413f-f096-4cf1-98ba-bd5b8a8dc8f7-kube-api-access-cpggl\") pod \"payload-processing-f9b8d9b9b-z6rq6\" (UID: \"baac413f-f096-4cf1-98ba-bd5b8a8dc8f7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"
Jul 14 17:14:16.683025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.682923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baac413f-f096-4cf1-98ba-bd5b8a8dc8f7-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-z6rq6\" (UID: \"baac413f-f096-4cf1-98ba-bd5b8a8dc8f7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"
Jul 14 17:14:16.683025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.682964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v865c\" (UniqueName: \"kubernetes.io/projected/7560cab2-d533-4a56-8a9d-497347e810de-kube-api-access-v865c\") pod \"payload-pre-processing-74584bb9f8-zsl6p\" (UID: \"7560cab2-d533-4a56-8a9d-497347e810de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"
Jul 14 17:14:16.683776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.683746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baac413f-f096-4cf1-98ba-bd5b8a8dc8f7-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-z6rq6\" (UID: \"baac413f-f096-4cf1-98ba-bd5b8a8dc8f7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"
Jul 14 17:14:16.683864 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.683746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7560cab2-d533-4a56-8a9d-497347e810de-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-zsl6p\" (UID: \"7560cab2-d533-4a56-8a9d-497347e810de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"
Jul 14 17:14:16.698261 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.698230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v865c\" (UniqueName: \"kubernetes.io/projected/7560cab2-d533-4a56-8a9d-497347e810de-kube-api-access-v865c\") pod \"payload-pre-processing-74584bb9f8-zsl6p\" (UID: \"7560cab2-d533-4a56-8a9d-497347e810de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"
Jul 14 17:14:16.698406 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.698276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpggl\" (UniqueName: \"kubernetes.io/projected/baac413f-f096-4cf1-98ba-bd5b8a8dc8f7-kube-api-access-cpggl\") pod \"payload-processing-f9b8d9b9b-z6rq6\" (UID: \"baac413f-f096-4cf1-98ba-bd5b8a8dc8f7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"
Jul 14 17:14:16.848366 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.848279    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"
Jul 14 17:14:16.873665 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.873622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"
Jul 14 17:14:16.985817 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:16.985780    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:14:17.001564 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:17.001537    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"]
Jul 14 17:14:17.002554 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:14:17.002520    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7560cab2_d533_4a56_8a9d_497347e810de.slice/crio-57455109b65747df655f9a4d965170ca108a329b61fb8e1b8127d70a789ca678 WatchSource:0}: Error finding container 57455109b65747df655f9a4d965170ca108a329b61fb8e1b8127d70a789ca678: Status 404 returned error can't find the container with id 57455109b65747df655f9a4d965170ca108a329b61fb8e1b8127d70a789ca678
Jul 14 17:14:17.032567 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:17.032398    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"]
Jul 14 17:14:17.035195 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:14:17.035165    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaac413f_f096_4cf1_98ba_bd5b8a8dc8f7.slice/crio-8200b1fd1bf219df68fe7788b562b28f929d2b2426c81d2acf051126c88a1dd0 WatchSource:0}: Error finding container 8200b1fd1bf219df68fe7788b562b28f929d2b2426c81d2acf051126c88a1dd0: Status 404 returned error can't find the container with id 8200b1fd1bf219df68fe7788b562b28f929d2b2426c81d2acf051126c88a1dd0
Jul 14 17:14:17.265565 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:17.265524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6" event={"ID":"baac413f-f096-4cf1-98ba-bd5b8a8dc8f7","Type":"ContainerStarted","Data":"8200b1fd1bf219df68fe7788b562b28f929d2b2426c81d2acf051126c88a1dd0"}
Jul 14 17:14:17.274872 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:17.267021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs" event={"ID":"ed7e0b20-692f-43a1-9e55-4aa569e4a031","Type":"ContainerStarted","Data":"09ea3f716253829d894dd1571edc4d65d035934d872c54116c2d9447e0fa9665"}
Jul 14 17:14:17.287024 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:17.286977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p" event={"ID":"7560cab2-d533-4a56-8a9d-497347e810de","Type":"ContainerStarted","Data":"57455109b65747df655f9a4d965170ca108a329b61fb8e1b8127d70a789ca678"}
Jul 14 17:14:19.308463 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:19.308370    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6" event={"ID":"baac413f-f096-4cf1-98ba-bd5b8a8dc8f7","Type":"ContainerStarted","Data":"1696da26716800984530e0c2f554946509ab8c7e75009e5477861f08edb7ab2f"}
Jul 14 17:14:19.308932 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:19.308543    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"
Jul 14 17:14:19.310212 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:19.310183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs" event={"ID":"ed7e0b20-692f-43a1-9e55-4aa569e4a031","Type":"ContainerStarted","Data":"7da05e14d9f61a57fb656558b9ba15cf055196f34f23a22a670f177138834f80"}
Jul 14 17:14:19.310340 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:19.310239    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:19.312065 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:19.312038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p" event={"ID":"7560cab2-d533-4a56-8a9d-497347e810de","Type":"ContainerStarted","Data":"caba4951fd32e1c187c2382bfc1b69e4cecaa0dab63a8522e29990e279e5b020"}
Jul 14 17:14:19.312187 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:19.312171    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"
Jul 14 17:14:19.331706 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:19.331659    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6" podStartSLOduration=1.316680225 podStartE2EDuration="3.331640388s" totalImagesPullingTime="2.014960163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:14:19.331262279 +0000 UTC m=+546.078862156" watchObservedRunningTime="2026-07-14 17:14:19.331640388 +0000 UTC m=+546.079240268"
Jul 14 17:14:19.355207 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:19.355159    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs" podStartSLOduration=1.777951486 podStartE2EDuration="4.355141864s" totalImagesPullingTime="2.577190378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:14:19.354273802 +0000 UTC m=+546.101873695" watchObservedRunningTime="2026-07-14 17:14:19.355141864 +0000 UTC m=+546.102741741"
Jul 14 17:14:19.376634 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:19.374215    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p" podStartSLOduration=1.324223498 podStartE2EDuration="3.374191043s" totalImagesPullingTime="2.049967545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:14:19.373135078 +0000 UTC m=+546.120734957" watchObservedRunningTime="2026-07-14 17:14:19.374191043 +0000 UTC m=+546.121790921"
Jul 14 17:14:24.336566 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:24.336528    2580 generic.go:356] "Generic (PLEG): container finished" podID="80fadef6-b849-463d-9ead-35c0ca1bc8cd" containerID="7e782c6254445f775abea5a5cf6fe0b3d69c389298db134b9f4bc1576463e94c" exitCode=0
Jul 14 17:14:24.337007 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:24.336602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw" event={"ID":"80fadef6-b849-463d-9ead-35c0ca1bc8cd","Type":"ContainerDied","Data":"7e782c6254445f775abea5a5cf6fe0b3d69c389298db134b9f4bc1576463e94c"}
Jul 14 17:14:24.337054 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:24.337006    2580 scope.go:117] "RemoveContainer" containerID="7e782c6254445f775abea5a5cf6fe0b3d69c389298db134b9f4bc1576463e94c"
Jul 14 17:14:24.337982 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:24.337953    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:14:24.338046 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:24.338020    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:14:24.338089 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:24.338053    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:14:25.322540 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:25.322508    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54675d56c-659bs"
Jul 14 17:14:25.344593 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:25.344558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw" event={"ID":"80fadef6-b849-463d-9ead-35c0ca1bc8cd","Type":"ContainerStarted","Data":"40bae3eb98b9f2b8a2befa5c9b3b48be29208a495e08f6a7adc84ea4f5ee7939"}
Jul 14 17:14:25.629169 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:25.629076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:14:25.630323 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:25.630294    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-nd7hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 14 17:14:25.630450 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:25.630345    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw" podUID="80fadef6-b849-463d-9ead-35c0ca1bc8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 14 17:14:26.629087 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:26.629049    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-nd7hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 14 17:14:26.629518 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:26.629110    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw" podUID="80fadef6-b849-463d-9ead-35c0ca1bc8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 14 17:14:27.628939 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:27.628904    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-nd7hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 14 17:14:27.629117 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:27.628966    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw" podUID="80fadef6-b849-463d-9ead-35c0ca1bc8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 14 17:14:28.629663 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:28.629622    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-nd7hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 14 17:14:28.630089 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:28.629680    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw" podUID="80fadef6-b849-463d-9ead-35c0ca1bc8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 14 17:14:28.890462 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:28.890352    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:14:29.632585 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:29.632553    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:14:29.633726 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:29.633708    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nd7hw"
Jul 14 17:14:30.319413 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:30.319384    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-z6rq6"
Jul 14 17:14:30.319590 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:30.319469    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-zsl6p"
Jul 14 17:14:43.148308 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.148270    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"]
Jul 14 17:14:43.151761 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.151737    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.155597 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.155573    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 14 17:14:43.155733 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.155599    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 14 17:14:43.155733 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.155581    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xt9hh\""
Jul 14 17:14:43.155733 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.155635    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 14 17:14:43.160108 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.160085    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"]
Jul 14 17:14:43.218333 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.218294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.218503 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.218374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.218503 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.218401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.218503 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.218435    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw7r7\" (UniqueName: \"kubernetes.io/projected/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-kube-api-access-cw7r7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.218503 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.218476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.218635 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.218513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.319594 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.319559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.319780 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.319602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.319780 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.319631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw7r7\" (UniqueName: \"kubernetes.io/projected/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-kube-api-access-cw7r7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.319780 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.319653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.319780 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.319680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.319780 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.319735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.320177 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.320154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.320250 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.320189    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.320250 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.320218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.322177 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.322148    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.322307 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.322281    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.330889 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.330863    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw7r7\" (UniqueName: \"kubernetes.io/projected/c9efe483-82ff-4ef4-b89a-dec4e3f1ffca-kube-api-access-cw7r7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk\" (UID: \"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.463046 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.463015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:43.604403 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.604379    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"]
Jul 14 17:14:43.606586 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:14:43.606553    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9efe483_82ff_4ef4_b89a_dec4e3f1ffca.slice/crio-b1184dd7b9d49533fd081ec2af3c56e7d72dd6b9899b3ee9717c602d0628e68e WatchSource:0}: Error finding container b1184dd7b9d49533fd081ec2af3c56e7d72dd6b9899b3ee9717c602d0628e68e: Status 404 returned error can't find the container with id b1184dd7b9d49533fd081ec2af3c56e7d72dd6b9899b3ee9717c602d0628e68e
Jul 14 17:14:43.728606 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:43.728523    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:14:44.439606 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:44.439565    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk" event={"ID":"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca","Type":"ContainerStarted","Data":"b1184dd7b9d49533fd081ec2af3c56e7d72dd6b9899b3ee9717c602d0628e68e"}
Jul 14 17:14:45.445760 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:45.445721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk" event={"ID":"c9efe483-82ff-4ef4-b89a-dec4e3f1ffca","Type":"ContainerStarted","Data":"686aeb14f38e0c16c7ac19d2ba8855121564a9232126c38a71782e2fe5bd540c"}
Jul 14 17:14:45.471563 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:45.471504    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk" podStartSLOduration=1.302525965 podStartE2EDuration="2.471483757s" totalImagesPullingTime="1.168957792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:14:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:14:45.46836663 +0000 UTC m=+572.215966507" watchObservedRunningTime="2026-07-14 17:14:45.471483757 +0000 UTC m=+572.219083634"
Jul 14 17:14:47.844441 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:47.844398    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:14:53.463772 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:53.463731    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:53.464198 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:53.463867    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:53.480237 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:53.480214    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:53.499598 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:53.499571    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk"
Jul 14 17:14:55.344416 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:14:55.344384    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:15:00.160588 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:00.160556    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734155-swwsx"]
Jul 14 17:15:00.166026 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:00.166008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734155-swwsx"
Jul 14 17:15:00.170183 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:00.170161    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\""
Jul 14 17:15:00.176662 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:00.176638    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734155-swwsx"
Jul 14 17:15:00.178393 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:00.178371    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734155-swwsx"]
Jul 14 17:15:00.315945 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:00.315921    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734155-swwsx"]
Jul 14 17:15:00.318929 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:15:00.318893    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13eb8d8d_88ac_4194_baf1_4b439e86d649.slice/crio-a7d7bdd1df3c2d7bc85e20d0f455c1f95909164f72051f28c57b7dad8fe78541 WatchSource:0}: Error finding container a7d7bdd1df3c2d7bc85e20d0f455c1f95909164f72051f28c57b7dad8fe78541: Status 404 returned error can't find the container with id a7d7bdd1df3c2d7bc85e20d0f455c1f95909164f72051f28c57b7dad8fe78541
Jul 14 17:15:00.520132 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:00.520099    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734155-swwsx" event={"ID":"13eb8d8d-88ac-4194-baf1-4b439e86d649","Type":"ContainerStarted","Data":"a7d7bdd1df3c2d7bc85e20d0f455c1f95909164f72051f28c57b7dad8fe78541"}
Jul 14 17:15:03.536728 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:03.536691    2580 generic.go:356] "Generic (PLEG): container finished" podID="13eb8d8d-88ac-4194-baf1-4b439e86d649" containerID="afccf1bbbae9daa164229327dd8852461be2128eb8c061ab3af21c67f7abfa71" exitCode=0
Jul 14 17:15:03.537145 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:03.536779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734155-swwsx" event={"ID":"13eb8d8d-88ac-4194-baf1-4b439e86d649","Type":"ContainerDied","Data":"afccf1bbbae9daa164229327dd8852461be2128eb8c061ab3af21c67f7abfa71"}
Jul 14 17:15:04.680648 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:04.680615    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734155-swwsx"
Jul 14 17:15:05.547754 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:05.547721    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734155-swwsx"
Jul 14 17:15:05.547953 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:05.547755    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734155-swwsx" event={"ID":"13eb8d8d-88ac-4194-baf1-4b439e86d649","Type":"ContainerDied","Data":"a7d7bdd1df3c2d7bc85e20d0f455c1f95909164f72051f28c57b7dad8fe78541"}
Jul 14 17:15:05.547953 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:05.547801    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7d7bdd1df3c2d7bc85e20d0f455c1f95909164f72051f28c57b7dad8fe78541"
Jul 14 17:15:13.701074 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:13.701044    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:15:13.701526 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:13.701141    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:15:29.763240 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:29.763208    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:15:32.457940 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.457908    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"]
Jul 14 17:15:32.458336 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.458288    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13eb8d8d-88ac-4194-baf1-4b439e86d649" containerName="cleanup"
Jul 14 17:15:32.458336 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.458300    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="13eb8d8d-88ac-4194-baf1-4b439e86d649" containerName="cleanup"
Jul 14 17:15:32.458419 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.458369    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="13eb8d8d-88ac-4194-baf1-4b439e86d649" containerName="cleanup"
Jul 14 17:15:32.461719 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.461697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.467526 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.467502    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 14 17:15:32.501692 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.501657    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"]
Jul 14 17:15:32.567150 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.567118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.567353 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.567156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.567353 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.567181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.567353 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.567255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.567353 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.567278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krsmb\" (UniqueName: \"kubernetes.io/projected/ff829063-494f-44ff-8d88-bc8a7e0270dd-kube-api-access-krsmb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.567353 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.567306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff829063-494f-44ff-8d88-bc8a7e0270dd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.667879 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.667806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krsmb\" (UniqueName: \"kubernetes.io/projected/ff829063-494f-44ff-8d88-bc8a7e0270dd-kube-api-access-krsmb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.668071 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.667899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff829063-494f-44ff-8d88-bc8a7e0270dd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.668071 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.667952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.668071 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.667974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.668071 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.667999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.668281 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.668071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.668404 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.668379    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.668578 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.668422    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.668651 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.668448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.671122 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.671093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff829063-494f-44ff-8d88-bc8a7e0270dd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.671369 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.671351    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff829063-494f-44ff-8d88-bc8a7e0270dd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.684667 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.684634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krsmb\" (UniqueName: \"kubernetes.io/projected/ff829063-494f-44ff-8d88-bc8a7e0270dd-kube-api-access-krsmb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f4wvb\" (UID: \"ff829063-494f-44ff-8d88-bc8a7e0270dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.771643 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.771546    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:32.920837 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.920811    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"]
Jul 14 17:15:32.922662 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:15:32.922634    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff829063_494f_44ff_8d88_bc8a7e0270dd.slice/crio-80039763d51d1b305df4c76199870cb1902f2f1ba00013bd554559802b64f200 WatchSource:0}: Error finding container 80039763d51d1b305df4c76199870cb1902f2f1ba00013bd554559802b64f200: Status 404 returned error can't find the container with id 80039763d51d1b305df4c76199870cb1902f2f1ba00013bd554559802b64f200
Jul 14 17:15:32.923988 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:32.923968    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 17:15:33.684210 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:33.684126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb" event={"ID":"ff829063-494f-44ff-8d88-bc8a7e0270dd","Type":"ContainerStarted","Data":"cd769989d42081eb1c87a86d18596c6eb60f918443e3d0b0a16b1517ba2d2239"}
Jul 14 17:15:33.684210 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:33.684181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb" event={"ID":"ff829063-494f-44ff-8d88-bc8a7e0270dd","Type":"ContainerStarted","Data":"80039763d51d1b305df4c76199870cb1902f2f1ba00013bd554559802b64f200"}
Jul 14 17:15:33.713913 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:33.713856    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb" podStartSLOduration=1.515205852 podStartE2EDuration="1.713827699s" totalImagesPullingTime="198.621847ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:15:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:15:33.7126872 +0000 UTC m=+620.460287078" watchObservedRunningTime="2026-07-14 17:15:33.713827699 +0000 UTC m=+620.461427576"
Jul 14 17:15:35.568056 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:35.568018    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:15:42.772544 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:42.772507    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:42.772963 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:42.772566    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:42.785089 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:42.785060    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:43.744069 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:43.744040    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f4wvb"
Jul 14 17:15:46.860151 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:15:46.860119    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:17:34.244561 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:17:34.244524    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:17:44.647977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:17:44.647931    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:17:55.453649 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:17:55.453611    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:18:03.786581 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:03.786543    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:18:14.936439 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:14.936398    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:18:17.026872 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.026823    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64846ff95c-b7964"]
Jul 14 17:18:17.027250 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.027029    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64846ff95c-b7964" podUID="75b9ec7c-b76d-4266-ae5f-576cf90aff9f" containerName="manager" containerID="cri-o://a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867" gracePeriod=30
Jul 14 17:18:17.273357 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.273330    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:18:17.448930 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.448883    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgzrd\" (UniqueName: \"kubernetes.io/projected/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-kube-api-access-qgzrd\") pod \"75b9ec7c-b76d-4266-ae5f-576cf90aff9f\" (UID: \"75b9ec7c-b76d-4266-ae5f-576cf90aff9f\") "
Jul 14 17:18:17.449123 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.448980    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-webhook-cert\") pod \"75b9ec7c-b76d-4266-ae5f-576cf90aff9f\" (UID: \"75b9ec7c-b76d-4266-ae5f-576cf90aff9f\") "
Jul 14 17:18:17.451263 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.451235    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "75b9ec7c-b76d-4266-ae5f-576cf90aff9f" (UID: "75b9ec7c-b76d-4266-ae5f-576cf90aff9f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:18:17.451354 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.451334    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-kube-api-access-qgzrd" (OuterVolumeSpecName: "kube-api-access-qgzrd") pod "75b9ec7c-b76d-4266-ae5f-576cf90aff9f" (UID: "75b9ec7c-b76d-4266-ae5f-576cf90aff9f"). InnerVolumeSpecName "kube-api-access-qgzrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:18:17.474790 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.474758    2580 generic.go:356] "Generic (PLEG): container finished" podID="75b9ec7c-b76d-4266-ae5f-576cf90aff9f" containerID="a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867" exitCode=0
Jul 14 17:18:17.474977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.474803    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64846ff95c-b7964"
Jul 14 17:18:17.474977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.474815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64846ff95c-b7964" event={"ID":"75b9ec7c-b76d-4266-ae5f-576cf90aff9f","Type":"ContainerDied","Data":"a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867"}
Jul 14 17:18:17.474977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.474898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64846ff95c-b7964" event={"ID":"75b9ec7c-b76d-4266-ae5f-576cf90aff9f","Type":"ContainerDied","Data":"aa860fc4c21f95fffe1927785bce0c153702d415ec27f0037ce88691834c7c74"}
Jul 14 17:18:17.474977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.474929    2580 scope.go:117] "RemoveContainer" containerID="a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867"
Jul 14 17:18:17.484004 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.483987    2580 scope.go:117] "RemoveContainer" containerID="a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867"
Jul 14 17:18:17.484254 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:18:17.484237    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867\": container with ID starting with a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867 not found: ID does not exist" containerID="a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867"
Jul 14 17:18:17.484307 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.484265    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867"} err="failed to get container status \"a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867\": rpc error: code = NotFound desc = could not find container \"a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867\": container with ID starting with a073db1fe9b6b070c35a801fbc9d81760c645a5bf7c101ce5d49995e2f210867 not found: ID does not exist"
Jul 14 17:18:17.500267 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.500234    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64846ff95c-b7964"]
Jul 14 17:18:17.512745 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.512706    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64846ff95c-b7964"]
Jul 14 17:18:17.550359 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.550320    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgzrd\" (UniqueName: \"kubernetes.io/projected/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-kube-api-access-qgzrd\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:18:17.550359 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.550355    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75b9ec7c-b76d-4266-ae5f-576cf90aff9f-webhook-cert\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:18:17.882683 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:17.882605    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75b9ec7c-b76d-4266-ae5f-576cf90aff9f" path="/var/lib/kubelet/pods/75b9ec7c-b76d-4266-ae5f-576cf90aff9f/volumes"
Jul 14 17:18:40.752799 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:40.752718    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:18:53.743498 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:18:53.743462    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:19:30.541948 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:19:30.541909    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:19:40.251125 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:19:40.251085    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:19:49.031819 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:19:49.031773    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:19:55.556703 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:19:55.556668    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:20:06.639402 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:20:06.639318    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:20:13.730802 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:20:13.730771    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:20:13.732062 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:20:13.732046    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:20:17.939346 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:20:17.939308    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:20:29.241695 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:20:29.241648    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:20:40.144256 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:20:40.144213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:21:04.593062 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:21:04.593028    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:21:16.010260 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:21:16.010222    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:21:38.750235 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:21:38.750146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:21:49.536934 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:21:49.536893    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:22:04.434694 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:22:04.434653    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:22:15.978237 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:22:15.978200    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:22:30.943371 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:22:30.943334    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:24:32.744764 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:24:32.744726    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:24:43.857142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:24:43.857050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:24:52.946181 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:24:52.946139    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:25:07.838239 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:25:07.838197    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:25:13.759406 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:25:13.759375    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:25:13.761628 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:25:13.761606    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:25:18.039088 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:25:18.039051    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:25:28.128813 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:25:28.128779    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:25:36.164199 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:25:36.164160    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:25:46.354284 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:25:46.354245    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:25:58.649352 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:25:58.649319    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:26:06.758289 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:26:06.758203    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:26:19.661053 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:26:19.661021    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:26:42.077598 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:26:42.077557    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:26:51.966471 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:26:51.966434    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:27:02.251314 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:27:02.251275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:27:12.274368 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:27:12.274327    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:27:21.668070 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:27:21.668028    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:27:34.464661 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:27:34.464568    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:27:44.844735 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:27:44.844698    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:27:54.848124 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:27:54.848079    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:28:05.740694 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:28:05.740652    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:28:15.088513 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:28:15.088477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:28:25.954948 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:28:25.954904    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:28:43.351429 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:28:43.351393    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:28:54.456474 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:28:54.456436    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:29:07.342476 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:29:07.342395    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:29:16.658204 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:29:16.658166    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:29:23.369673 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:29:23.369634    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:29:40.853666 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:29:40.853619    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:29:55.560765 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:29:55.560730    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:30:08.254012 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:08.253971    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:30:13.788516 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:13.788476    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:30:13.790721 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:13.790702    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:30:21.536058 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:21.536021    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:30:34.743389 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:34.743302    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:30:43.200166 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.200131    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"]
Jul 14 17:30:43.200615 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.200547    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75b9ec7c-b76d-4266-ae5f-576cf90aff9f" containerName="manager"
Jul 14 17:30:43.200615 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.200559    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="75b9ec7c-b76d-4266-ae5f-576cf90aff9f" containerName="manager"
Jul 14 17:30:43.200615 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.200618    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="75b9ec7c-b76d-4266-ae5f-576cf90aff9f" containerName="manager"
Jul 14 17:30:43.202618 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.202593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:43.205998 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.205968    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fa58286c-serving-cert\""
Jul 14 17:30:43.220965 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.220938    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"]
Jul 14 17:30:43.274747 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.274711    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xrgr\" (UniqueName: \"kubernetes.io/projected/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-kube-api-access-8xrgr\") pod \"maas-api-e2e-ait-fa58286c-545f59cb49-8nndl\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:43.274940 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.274775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-maas-api-tls\") pod \"maas-api-e2e-ait-fa58286c-545f59cb49-8nndl\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:43.274940 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.274861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-openshift-service-ca\") pod \"maas-api-e2e-ait-fa58286c-545f59cb49-8nndl\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:43.376083 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.376044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-maas-api-tls\") pod \"maas-api-e2e-ait-fa58286c-545f59cb49-8nndl\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:43.376246 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.376091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-openshift-service-ca\") pod \"maas-api-e2e-ait-fa58286c-545f59cb49-8nndl\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:43.376246 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.376146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xrgr\" (UniqueName: \"kubernetes.io/projected/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-kube-api-access-8xrgr\") pod \"maas-api-e2e-ait-fa58286c-545f59cb49-8nndl\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:43.376246 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:30:43.376224    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-serving-cert: secret "maas-api-e2e-ait-fa58286c-serving-cert" not found
Jul 14 17:30:43.376352 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:30:43.376306    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-maas-api-tls podName:b37cc894-a2d3-4508-90dc-5f6a72fa1ffd nodeName:}" failed. No retries permitted until 2026-07-14 17:30:43.876283493 +0000 UTC m=+1530.623883368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-maas-api-tls") pod "maas-api-e2e-ait-fa58286c-545f59cb49-8nndl" (UID: "b37cc894-a2d3-4508-90dc-5f6a72fa1ffd") : secret "maas-api-e2e-ait-fa58286c-serving-cert" not found
Jul 14 17:30:43.376810 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.376790    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-openshift-service-ca\") pod \"maas-api-e2e-ait-fa58286c-545f59cb49-8nndl\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:43.386876 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.386828    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xrgr\" (UniqueName: \"kubernetes.io/projected/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-kube-api-access-8xrgr\") pod \"maas-api-e2e-ait-fa58286c-545f59cb49-8nndl\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:43.880267 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.880234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-maas-api-tls\") pod \"maas-api-e2e-ait-fa58286c-545f59cb49-8nndl\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:43.883121 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:43.883099    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-maas-api-tls\") pod \"maas-api-e2e-ait-fa58286c-545f59cb49-8nndl\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:44.114146 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:44.114114    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:44.255467 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:44.255428    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"]
Jul 14 17:30:44.258943 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:30:44.258904    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb37cc894_a2d3_4508_90dc_5f6a72fa1ffd.slice/crio-6009f8c8d33db97325978c9a0fea949df02f1e9d2b91fb0dc3395407a74508b2 WatchSource:0}: Error finding container 6009f8c8d33db97325978c9a0fea949df02f1e9d2b91fb0dc3395407a74508b2: Status 404 returned error can't find the container with id 6009f8c8d33db97325978c9a0fea949df02f1e9d2b91fb0dc3395407a74508b2
Jul 14 17:30:44.260322 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:44.260300    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 17:30:45.048716 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:45.048673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl" event={"ID":"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd","Type":"ContainerStarted","Data":"532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a"}
Jul 14 17:30:45.048941 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:45.048740    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl" event={"ID":"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd","Type":"ContainerStarted","Data":"6009f8c8d33db97325978c9a0fea949df02f1e9d2b91fb0dc3395407a74508b2"}
Jul 14 17:30:45.048941 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:45.048768    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:30:45.070735 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:45.070686    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl" podStartSLOduration=1.508116238 podStartE2EDuration="2.070668332s" totalImagesPullingTime="562.552094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:30:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:30:45.068324601 +0000 UTC m=+1531.815924494" watchObservedRunningTime="2026-07-14 17:30:45.070668332 +0000 UTC m=+1531.818268506"
Jul 14 17:30:50.735075 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:50.735031    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:30:51.058527 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:30:51.058447    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:31:58.697472 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:58.697435    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"]
Jul 14 17:31:58.698123 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:58.697688    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl" podUID="b37cc894-a2d3-4508-90dc-5f6a72fa1ffd" containerName="maas-api" containerID="cri-o://532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a" gracePeriod=30
Jul 14 17:31:58.949438 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:58.949375    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:31:59.053475 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.053428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-maas-api-tls\") pod \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") "
Jul 14 17:31:59.053665 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.053502    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-openshift-service-ca\") pod \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") "
Jul 14 17:31:59.053665 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.053532    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xrgr\" (UniqueName: \"kubernetes.io/projected/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-kube-api-access-8xrgr\") pod \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\" (UID: \"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd\") "
Jul 14 17:31:59.053899 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.053872    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b37cc894-a2d3-4508-90dc-5f6a72fa1ffd" (UID: "b37cc894-a2d3-4508-90dc-5f6a72fa1ffd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:31:59.055812 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.055780    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b37cc894-a2d3-4508-90dc-5f6a72fa1ffd" (UID: "b37cc894-a2d3-4508-90dc-5f6a72fa1ffd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:31:59.055943 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.055832    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-kube-api-access-8xrgr" (OuterVolumeSpecName: "kube-api-access-8xrgr") pod "b37cc894-a2d3-4508-90dc-5f6a72fa1ffd" (UID: "b37cc894-a2d3-4508-90dc-5f6a72fa1ffd"). InnerVolumeSpecName "kube-api-access-8xrgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:31:59.155132 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.155095    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-maas-api-tls\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:31:59.155132 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.155125    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-openshift-service-ca\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:31:59.155132 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.155136    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xrgr\" (UniqueName: \"kubernetes.io/projected/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd-kube-api-access-8xrgr\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:31:59.399025 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.398883    2580 generic.go:356] "Generic (PLEG): container finished" podID="b37cc894-a2d3-4508-90dc-5f6a72fa1ffd" containerID="532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a" exitCode=0
Jul 14 17:31:59.399357 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.399293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl" event={"ID":"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd","Type":"ContainerDied","Data":"532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a"}
Jul 14 17:31:59.399575 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.399520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl" event={"ID":"b37cc894-a2d3-4508-90dc-5f6a72fa1ffd","Type":"ContainerDied","Data":"6009f8c8d33db97325978c9a0fea949df02f1e9d2b91fb0dc3395407a74508b2"}
Jul 14 17:31:59.399575 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.399550    2580 scope.go:117] "RemoveContainer" containerID="532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a"
Jul 14 17:31:59.399694 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.399618    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"
Jul 14 17:31:59.408824 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.408800    2580 scope.go:117] "RemoveContainer" containerID="532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a"
Jul 14 17:31:59.409124 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:31:59.409102    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a\": container with ID starting with 532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a not found: ID does not exist" containerID="532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a"
Jul 14 17:31:59.409182 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.409133    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a"} err="failed to get container status \"532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a\": rpc error: code = NotFound desc = could not find container \"532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a\": container with ID starting with 532ba428c61aebebd46e4243e1355e483cc862a1ff951f2a84ca7d0b51529b5a not found: ID does not exist"
Jul 14 17:31:59.424318 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.424290    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"]
Jul 14 17:31:59.429547 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.429523    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa58286c-545f59cb49-8nndl"]
Jul 14 17:31:59.882361 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:31:59.882326    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b37cc894-a2d3-4508-90dc-5f6a72fa1ffd" path="/var/lib/kubelet/pods/b37cc894-a2d3-4508-90dc-5f6a72fa1ffd/volumes"
Jul 14 17:32:00.540232 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:00.540190    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:32:13.993961 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:13.993929    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"]
Jul 14 17:32:13.994358 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:13.994274    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b37cc894-a2d3-4508-90dc-5f6a72fa1ffd" containerName="maas-api"
Jul 14 17:32:13.994358 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:13.994286    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b37cc894-a2d3-4508-90dc-5f6a72fa1ffd" containerName="maas-api"
Jul 14 17:32:13.994358 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:13.994357    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b37cc894-a2d3-4508-90dc-5f6a72fa1ffd" containerName="maas-api"
Jul 14 17:32:13.998429 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:13.998411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.001124 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.001101    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-288a19fd-gw-openshift-default-dockercfg-r4ccv\""
Jul 14 17:32:14.011019 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.010992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"]
Jul 14 17:32:14.093756 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.093713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-envoy\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.093971 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.093766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-istio-token\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.093971 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.093799    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf6520f3-b706-423e-955c-d0a06546033d-istiod-ca-cert\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.093971 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.093825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-credential-socket\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.093971 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.093867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-socket\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.093971 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.093912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w24dx\" (UniqueName: \"kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-kube-api-access-w24dx\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.094146 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.093965    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-data\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.094146 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.094019    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-certs\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.094146 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.094067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf6520f3-b706-423e-955c-d0a06546033d-istio-podinfo\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.194625 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.194593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w24dx\" (UniqueName: \"kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-kube-api-access-w24dx\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.194784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.194632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-data\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.194784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.194653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-certs\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.194784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.194683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf6520f3-b706-423e-955c-d0a06546033d-istio-podinfo\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.194784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.194721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-envoy\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.194784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.194753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-istio-token\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.194784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.194780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf6520f3-b706-423e-955c-d0a06546033d-istiod-ca-cert\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.195124 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.194834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-credential-socket\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.195124 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.195050    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-certs\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.195124 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.195073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-data\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.195124 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.195075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-socket\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.195328 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.195223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-credential-socket\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.195391 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.195377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-socket\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.195476 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.195458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf6520f3-b706-423e-955c-d0a06546033d-istiod-ca-cert\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.197409 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.197378    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf6520f3-b706-423e-955c-d0a06546033d-istio-podinfo\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.197533 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.197447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-envoy\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.204576 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.204553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-istio-token\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.204702 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.204652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w24dx\" (UniqueName: \"kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-kube-api-access-w24dx\") pod \"e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") " pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.309249 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.309212    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:14.460878 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.460820    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"]
Jul 14 17:32:14.462745 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:32:14.462716    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf6520f3_b706_423e_955c_d0a06546033d.slice/crio-751e8001841fb5e291e9d456eb4897dc26f6587dc2f4c7fbb94702350bc9b7b8 WatchSource:0}: Error finding container 751e8001841fb5e291e9d456eb4897dc26f6587dc2f4c7fbb94702350bc9b7b8: Status 404 returned error can't find the container with id 751e8001841fb5e291e9d456eb4897dc26f6587dc2f4c7fbb94702350bc9b7b8
Jul 14 17:32:14.465093 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.465044    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:32:14.465174 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.465138    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:32:14.465229 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.465180    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:32:14.473885 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:14.473837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" event={"ID":"bf6520f3-b706-423e-955c-d0a06546033d","Type":"ContainerStarted","Data":"751e8001841fb5e291e9d456eb4897dc26f6587dc2f4c7fbb94702350bc9b7b8"}
Jul 14 17:32:15.343567 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.343533    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"]
Jul 14 17:32:15.347124 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.347099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.350517 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.350498    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-288a19fd-serving-cert\""
Jul 14 17:32:15.362968 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.362942    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"]
Jul 14 17:32:15.405872 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.405808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9125c06-629d-47ae-b7c9-629fef1590ea-maas-api-tls\") pod \"maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.406040 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.405922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9125c06-629d-47ae-b7c9-629fef1590ea-openshift-service-ca\") pod \"maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.406040 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.406004    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc4pt\" (UniqueName: \"kubernetes.io/projected/a9125c06-629d-47ae-b7c9-629fef1590ea-kube-api-access-kc4pt\") pod \"maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.483234 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.483195    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" event={"ID":"bf6520f3-b706-423e-955c-d0a06546033d","Type":"ContainerStarted","Data":"8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781"}
Jul 14 17:32:15.506699 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.506664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9125c06-629d-47ae-b7c9-629fef1590ea-openshift-service-ca\") pod \"maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.506916 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.506795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc4pt\" (UniqueName: \"kubernetes.io/projected/a9125c06-629d-47ae-b7c9-629fef1590ea-kube-api-access-kc4pt\") pod \"maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.506916 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.506888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9125c06-629d-47ae-b7c9-629fef1590ea-maas-api-tls\") pod \"maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.507470 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.507438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9125c06-629d-47ae-b7c9-629fef1590ea-openshift-service-ca\") pod \"maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.509790 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.509768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9125c06-629d-47ae-b7c9-629fef1590ea-maas-api-tls\") pod \"maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.509904 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.509816    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" podStartSLOduration=2.509801289 podStartE2EDuration="2.509801289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:32:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:15.50829686 +0000 UTC m=+1622.255896738" watchObservedRunningTime="2026-07-14 17:32:15.509801289 +0000 UTC m=+1622.257401242"
Jul 14 17:32:15.517636 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.517609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc4pt\" (UniqueName: \"kubernetes.io/projected/a9125c06-629d-47ae-b7c9-629fef1590ea-kube-api-access-kc4pt\") pod \"maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.657697 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.657601    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:15.805355 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:15.805328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"]
Jul 14 17:32:15.807416 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:32:15.807384    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9125c06_629d_47ae_b7c9_629fef1590ea.slice/crio-01a32f9bdf48487a034f36908551f4aaee3eb85ec9978f300d18a7ede0b50956 WatchSource:0}: Error finding container 01a32f9bdf48487a034f36908551f4aaee3eb85ec9978f300d18a7ede0b50956: Status 404 returned error can't find the container with id 01a32f9bdf48487a034f36908551f4aaee3eb85ec9978f300d18a7ede0b50956
Jul 14 17:32:16.309454 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:16.309412    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:16.314223 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:16.314194    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:16.489232 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:16.489205    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4" event={"ID":"a9125c06-629d-47ae-b7c9-629fef1590ea","Type":"ContainerStarted","Data":"01a32f9bdf48487a034f36908551f4aaee3eb85ec9978f300d18a7ede0b50956"}
Jul 14 17:32:16.489600 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:16.489528    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:16.490429 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:16.490408    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:17.495790 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:17.495754    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4" event={"ID":"a9125c06-629d-47ae-b7c9-629fef1590ea","Type":"ContainerStarted","Data":"c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b"}
Jul 14 17:32:17.519106 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:17.519053    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4" podStartSLOduration=1.842899863 podStartE2EDuration="2.519034901s" totalImagesPullingTime="676.135038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:17.517695187 +0000 UTC m=+1624.265295073" watchObservedRunningTime="2026-07-14 17:32:17.519034901 +0000 UTC m=+1624.266634809"
Jul 14 17:32:18.500465 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:18.500434    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:20.742573 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:20.742535    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:32:24.511213 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:24.511185    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:32:28.556318 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:28.556280    2580 generic.go:356] "Generic (PLEG): container finished" podID="bf6520f3-b706-423e-955c-d0a06546033d" containerID="8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781" exitCode=0
Jul 14 17:32:28.556836 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:28.556327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" event={"ID":"bf6520f3-b706-423e-955c-d0a06546033d","Type":"ContainerDied","Data":"8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781"}
Jul 14 17:32:28.556836 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:28.556820    2580 scope.go:117] "RemoveContainer" containerID="8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781"
Jul 14 17:32:28.557934 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:28.557905    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:32:28.558008 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:28.557975    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:32:28.558044 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:28.558008    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:32:29.562564 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:29.562530    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" event={"ID":"bf6520f3-b706-423e-955c-d0a06546033d","Type":"ContainerStarted","Data":"6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136"}
Jul 14 17:32:30.310407 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:30.310373    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:30.311459 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:30.311430    2580 patch_prober.go:28] interesting pod/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 14 17:32:30.311588 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:30.311490    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 14 17:32:31.309907 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:31.309830    2580 patch_prober.go:28] interesting pod/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 14 17:32:31.310501 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:31.309938    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 14 17:32:31.490232 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:31.490180    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:32.310484 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:32.310445    2580 patch_prober.go:28] interesting pod/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 14 17:32:32.310943 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:32.310507    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 14 17:32:33.310621 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:33.310581    2580 patch_prober.go:28] interesting pod/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 14 17:32:33.311046 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:33.310660    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 14 17:32:34.314231 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:34.314197    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:32:34.315269 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:32:34.315251    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:33:19.922718 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:19.922685    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"]
Jul 14 17:33:19.923215 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:19.922912    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy" containerID="cri-o://6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136" gracePeriod=30
Jul 14 17:33:22.862940 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:22.862904    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:33:25.174118 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.174093    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:33:25.250399 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250367    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-socket\") pod \"bf6520f3-b706-423e-955c-d0a06546033d\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") "
Jul 14 17:33:25.250399 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250402    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-certs\") pod \"bf6520f3-b706-423e-955c-d0a06546033d\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") "
Jul 14 17:33:25.250623 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250438    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w24dx\" (UniqueName: \"kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-kube-api-access-w24dx\") pod \"bf6520f3-b706-423e-955c-d0a06546033d\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") "
Jul 14 17:33:25.250623 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250459    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-credential-socket\") pod \"bf6520f3-b706-423e-955c-d0a06546033d\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") "
Jul 14 17:33:25.250623 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250500    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-istio-token\") pod \"bf6520f3-b706-423e-955c-d0a06546033d\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") "
Jul 14 17:33:25.250623 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250545    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-data\") pod \"bf6520f3-b706-423e-955c-d0a06546033d\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") "
Jul 14 17:33:25.250623 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250573    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf6520f3-b706-423e-955c-d0a06546033d-istio-podinfo\") pod \"bf6520f3-b706-423e-955c-d0a06546033d\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") "
Jul 14 17:33:25.250623 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250622    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-envoy\") pod \"bf6520f3-b706-423e-955c-d0a06546033d\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") "
Jul 14 17:33:25.250943 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250648    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf6520f3-b706-423e-955c-d0a06546033d-istiod-ca-cert\") pod \"bf6520f3-b706-423e-955c-d0a06546033d\" (UID: \"bf6520f3-b706-423e-955c-d0a06546033d\") "
Jul 14 17:33:25.250943 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250680    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bf6520f3-b706-423e-955c-d0a06546033d" (UID: "bf6520f3-b706-423e-955c-d0a06546033d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:33:25.250943 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250695    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bf6520f3-b706-423e-955c-d0a06546033d" (UID: "bf6520f3-b706-423e-955c-d0a06546033d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:33:25.250943 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250880    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bf6520f3-b706-423e-955c-d0a06546033d" (UID: "bf6520f3-b706-423e-955c-d0a06546033d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:33:25.251135 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250962    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-credential-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:25.251135 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250975    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:25.251135 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.250985    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-workload-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:25.251658 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.251617    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-data" (OuterVolumeSpecName: "istio-data") pod "bf6520f3-b706-423e-955c-d0a06546033d" (UID: "bf6520f3-b706-423e-955c-d0a06546033d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:33:25.251763 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.251736    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf6520f3-b706-423e-955c-d0a06546033d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bf6520f3-b706-423e-955c-d0a06546033d" (UID: "bf6520f3-b706-423e-955c-d0a06546033d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:33:25.253431 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.253407    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bf6520f3-b706-423e-955c-d0a06546033d" (UID: "bf6520f3-b706-423e-955c-d0a06546033d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:33:25.253530 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.253423    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-kube-api-access-w24dx" (OuterVolumeSpecName: "kube-api-access-w24dx") pod "bf6520f3-b706-423e-955c-d0a06546033d" (UID: "bf6520f3-b706-423e-955c-d0a06546033d"). InnerVolumeSpecName "kube-api-access-w24dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:33:25.253530 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.253440    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf6520f3-b706-423e-955c-d0a06546033d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bf6520f3-b706-423e-955c-d0a06546033d" (UID: "bf6520f3-b706-423e-955c-d0a06546033d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 17:33:25.253754 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.253720    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-istio-token" (OuterVolumeSpecName: "istio-token") pod "bf6520f3-b706-423e-955c-d0a06546033d" (UID: "bf6520f3-b706-423e-955c-d0a06546033d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:33:25.351643 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.351612    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-envoy\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:25.351643 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.351638    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf6520f3-b706-423e-955c-d0a06546033d-istiod-ca-cert\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:25.351643 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.351647    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w24dx\" (UniqueName: \"kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-kube-api-access-w24dx\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:25.351915 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.351656    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf6520f3-b706-423e-955c-d0a06546033d-istio-token\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:25.351915 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.351664    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf6520f3-b706-423e-955c-d0a06546033d-istio-data\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:25.351915 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.351672    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf6520f3-b706-423e-955c-d0a06546033d-istio-podinfo\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:25.850443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.850410    2580 generic.go:356] "Generic (PLEG): container finished" podID="bf6520f3-b706-423e-955c-d0a06546033d" containerID="6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136" exitCode=0
Jul 14 17:33:25.850635 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.850474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" event={"ID":"bf6520f3-b706-423e-955c-d0a06546033d","Type":"ContainerDied","Data":"6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136"}
Jul 14 17:33:25.850635 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.850531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz" event={"ID":"bf6520f3-b706-423e-955c-d0a06546033d","Type":"ContainerDied","Data":"751e8001841fb5e291e9d456eb4897dc26f6587dc2f4c7fbb94702350bc9b7b8"}
Jul 14 17:33:25.850635 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.850475    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"
Jul 14 17:33:25.850635 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.850566    2580 scope.go:117] "RemoveContainer" containerID="6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136"
Jul 14 17:33:25.862455 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.862432    2580 scope.go:117] "RemoveContainer" containerID="8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781"
Jul 14 17:33:25.863742 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.863719    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"]
Jul 14 17:33:25.863958 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.863937    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4" podUID="a9125c06-629d-47ae-b7c9-629fef1590ea" containerName="maas-api" containerID="cri-o://c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b" gracePeriod=30
Jul 14 17:33:25.874100 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.873935    2580 scope.go:117] "RemoveContainer" containerID="6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136"
Jul 14 17:33:25.874463 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:33:25.874432    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136\": container with ID starting with 6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136 not found: ID does not exist" containerID="6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136"
Jul 14 17:33:25.874554 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.874478    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136"} err="failed to get container status \"6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136\": rpc error: code = NotFound desc = could not find container \"6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136\": container with ID starting with 6fbcc71bbd4a0644f2310cbe61da304a9104230f6d95731d758a4d5d0c4af136 not found: ID does not exist"
Jul 14 17:33:25.874554 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.874505    2580 scope.go:117] "RemoveContainer" containerID="8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781"
Jul 14 17:33:25.874974 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:33:25.874948    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781\": container with ID starting with 8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781 not found: ID does not exist" containerID="8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781"
Jul 14 17:33:25.875055 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.874984    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781"} err="failed to get container status \"8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781\": rpc error: code = NotFound desc = could not find container \"8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781\": container with ID starting with 8516ba6f6b32bf948032a5ff6514b34169621542a708c99a662a656c0acdd781 not found: ID does not exist"
Jul 14 17:33:25.884611 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.884584    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"]
Jul 14 17:33:25.887495 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:25.887467    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-288a19fd-gw-openshift-default-7f8b4d4687-924hz"]
Jul 14 17:33:26.124539 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.124514    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:33:26.259081 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.259038    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9125c06-629d-47ae-b7c9-629fef1590ea-maas-api-tls\") pod \"a9125c06-629d-47ae-b7c9-629fef1590ea\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") "
Jul 14 17:33:26.259465 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.259159    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9125c06-629d-47ae-b7c9-629fef1590ea-openshift-service-ca\") pod \"a9125c06-629d-47ae-b7c9-629fef1590ea\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") "
Jul 14 17:33:26.259465 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.259208    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kc4pt\" (UniqueName: \"kubernetes.io/projected/a9125c06-629d-47ae-b7c9-629fef1590ea-kube-api-access-kc4pt\") pod \"a9125c06-629d-47ae-b7c9-629fef1590ea\" (UID: \"a9125c06-629d-47ae-b7c9-629fef1590ea\") "
Jul 14 17:33:26.259608 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.259567    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9125c06-629d-47ae-b7c9-629fef1590ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9125c06-629d-47ae-b7c9-629fef1590ea" (UID: "a9125c06-629d-47ae-b7c9-629fef1590ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:33:26.261387 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.261358    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9125c06-629d-47ae-b7c9-629fef1590ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9125c06-629d-47ae-b7c9-629fef1590ea" (UID: "a9125c06-629d-47ae-b7c9-629fef1590ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:33:26.261456 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.261446    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9125c06-629d-47ae-b7c9-629fef1590ea-kube-api-access-kc4pt" (OuterVolumeSpecName: "kube-api-access-kc4pt") pod "a9125c06-629d-47ae-b7c9-629fef1590ea" (UID: "a9125c06-629d-47ae-b7c9-629fef1590ea"). InnerVolumeSpecName "kube-api-access-kc4pt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:33:26.360649 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.360546    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9125c06-629d-47ae-b7c9-629fef1590ea-openshift-service-ca\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:26.360649 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.360584    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kc4pt\" (UniqueName: \"kubernetes.io/projected/a9125c06-629d-47ae-b7c9-629fef1590ea-kube-api-access-kc4pt\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:26.360649 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.360598    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9125c06-629d-47ae-b7c9-629fef1590ea-maas-api-tls\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:33:26.857533 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.857495    2580 generic.go:356] "Generic (PLEG): container finished" podID="a9125c06-629d-47ae-b7c9-629fef1590ea" containerID="c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b" exitCode=0
Jul 14 17:33:26.857738 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.857543    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"
Jul 14 17:33:26.857738 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.857597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4" event={"ID":"a9125c06-629d-47ae-b7c9-629fef1590ea","Type":"ContainerDied","Data":"c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b"}
Jul 14 17:33:26.857738 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.857666    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4" event={"ID":"a9125c06-629d-47ae-b7c9-629fef1590ea","Type":"ContainerDied","Data":"01a32f9bdf48487a034f36908551f4aaee3eb85ec9978f300d18a7ede0b50956"}
Jul 14 17:33:26.857738 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.857699    2580 scope.go:117] "RemoveContainer" containerID="c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b"
Jul 14 17:33:26.869573 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.869551    2580 scope.go:117] "RemoveContainer" containerID="c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b"
Jul 14 17:33:26.870092 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:33:26.870055    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b\": container with ID starting with c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b not found: ID does not exist" containerID="c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b"
Jul 14 17:33:26.870224 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.870108    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b"} err="failed to get container status \"c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b\": rpc error: code = NotFound desc = could not find container \"c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b\": container with ID starting with c0406efb6c7151250b7dfe7f22ae22321d42284fe1d9e67d107969efe1a13f8b not found: ID does not exist"
Jul 14 17:33:26.883522 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.883495    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"]
Jul 14 17:33:26.888515 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:26.888490    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-288a19fd-79c768bf74-6zxr4"]
Jul 14 17:33:27.881589 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:27.881555    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9125c06-629d-47ae-b7c9-629fef1590ea" path="/var/lib/kubelet/pods/a9125c06-629d-47ae-b7c9-629fef1590ea/volumes"
Jul 14 17:33:27.881998 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:27.881983    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf6520f3-b706-423e-955c-d0a06546033d" path="/var/lib/kubelet/pods/bf6520f3-b706-423e-955c-d0a06546033d/volumes"
Jul 14 17:33:33.000866 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.000817    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"]
Jul 14 17:33:33.001418 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.001389    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy"
Jul 14 17:33:33.001418 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.001417    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy"
Jul 14 17:33:33.001572 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.001433    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy"
Jul 14 17:33:33.001572 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.001439    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy"
Jul 14 17:33:33.001572 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.001453    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9125c06-629d-47ae-b7c9-629fef1590ea" containerName="maas-api"
Jul 14 17:33:33.001572 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.001459    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9125c06-629d-47ae-b7c9-629fef1590ea" containerName="maas-api"
Jul 14 17:33:33.001572 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.001514    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy"
Jul 14 17:33:33.001572 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.001521    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9125c06-629d-47ae-b7c9-629fef1590ea" containerName="maas-api"
Jul 14 17:33:33.006065 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.006043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.014050 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.014030    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-30670f0b-openshift-default-dockercfg-62fsp\""
Jul 14 17:33:33.080122 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.080083    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"]
Jul 14 17:33:33.125518 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.125478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-credential-socket\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.125518 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.125523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-data\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.125724 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.125544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-token\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.125724 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.125637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istiod-ca-cert\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.125724 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.125671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4ckv\" (UniqueName: \"kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-kube-api-access-n4ckv\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.125724 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.125690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-envoy\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.125886 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.125743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-socket\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.125886 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.125771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-certs\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.125886 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.125838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-podinfo\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227030 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.226935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-podinfo\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227030 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-credential-socket\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227254 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-data\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227254 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-token\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227254 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istiod-ca-cert\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227254 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4ckv\" (UniqueName: \"kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-kube-api-access-n4ckv\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227254 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-envoy\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227254 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-socket\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227554 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-credential-socket\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227554 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-certs\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227554 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-data\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227680 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227592    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-socket\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.227718 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.227696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-certs\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.228136 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.228109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istiod-ca-cert\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.229575 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.229553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-envoy\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.229688 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.229660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-podinfo\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.238353 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.238331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4ckv\" (UniqueName: \"kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-kube-api-access-n4ckv\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.238641 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.238620    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-token\") pod \"e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") " pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.316461 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.316421    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:33.464714 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.464685    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"]
Jul 14 17:33:33.467380 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:33:33.467347    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546b39ac_2e8a_45f0_a150_c6d14e944c3c.slice/crio-324650ca0f974a44271ed4dab72eaac28c17d68f5fb8b3310b379a4f830cf9bb WatchSource:0}: Error finding container 324650ca0f974a44271ed4dab72eaac28c17d68f5fb8b3310b379a4f830cf9bb: Status 404 returned error can't find the container with id 324650ca0f974a44271ed4dab72eaac28c17d68f5fb8b3310b379a4f830cf9bb
Jul 14 17:33:33.469775 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.469741    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:33:33.469898 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.469813    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:33:33.469898 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.469872    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:33:33.895597 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.895473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" event={"ID":"546b39ac-2e8a-45f0-a150-c6d14e944c3c","Type":"ContainerStarted","Data":"f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d"}
Jul 14 17:33:33.895597 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.895536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" event={"ID":"546b39ac-2e8a-45f0-a150-c6d14e944c3c","Type":"ContainerStarted","Data":"324650ca0f974a44271ed4dab72eaac28c17d68f5fb8b3310b379a4f830cf9bb"}
Jul 14 17:33:33.924879 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:33.924808    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" podStartSLOduration=1.924793181 podStartE2EDuration="1.924793181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:33.922329866 +0000 UTC m=+1700.669929755" watchObservedRunningTime="2026-07-14 17:33:33.924793181 +0000 UTC m=+1700.672393058"
Jul 14 17:33:34.314297 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.314263    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"]
Jul 14 17:33:34.314893 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.314872    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf6520f3-b706-423e-955c-d0a06546033d" containerName="istio-proxy"
Jul 14 17:33:34.317779 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.317757    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:34.317924 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.317878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.321058 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.321026    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-30670f0b-serving-cert\""
Jul 14 17:33:34.323375 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.323343    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:34.330264 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.330240    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"]
Jul 14 17:33:34.437781 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.437746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9h6p\" (UniqueName: \"kubernetes.io/projected/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-kube-api-access-c9h6p\") pod \"maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.437954 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.437795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-maas-api-tls\") pod \"maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.438021 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.437947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-openshift-service-ca\") pod \"maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.539269 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.539224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9h6p\" (UniqueName: \"kubernetes.io/projected/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-kube-api-access-c9h6p\") pod \"maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.539269 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.539275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-maas-api-tls\") pod \"maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.539518 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.539347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-openshift-service-ca\") pod \"maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.540090 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.540060    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-openshift-service-ca\") pod \"maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.541944 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.541923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-maas-api-tls\") pod \"maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.557588 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.557550    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9h6p\" (UniqueName: \"kubernetes.io/projected/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-kube-api-access-c9h6p\") pod \"maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.630447 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.630356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:34.776648 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.776593    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"]
Jul 14 17:33:34.778835 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:33:34.778808    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41bd93c_41b9_49ec_bf01_1b33262b0a9d.slice/crio-e616e9e3fd83533712369e2f6334d18ae33c96bad756f0a9bae68f2bf95ca585 WatchSource:0}: Error finding container e616e9e3fd83533712369e2f6334d18ae33c96bad756f0a9bae68f2bf95ca585: Status 404 returned error can't find the container with id e616e9e3fd83533712369e2f6334d18ae33c96bad756f0a9bae68f2bf95ca585
Jul 14 17:33:34.902064 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.901964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8" event={"ID":"d41bd93c-41b9-49ec-bf01-1b33262b0a9d","Type":"ContainerStarted","Data":"e616e9e3fd83533712369e2f6334d18ae33c96bad756f0a9bae68f2bf95ca585"}
Jul 14 17:33:34.902221 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.902190    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:34.903125 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:34.903108    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:35.907698 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:35.907657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8" event={"ID":"d41bd93c-41b9-49ec-bf01-1b33262b0a9d","Type":"ContainerStarted","Data":"302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c"}
Jul 14 17:33:35.908109 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:35.907756    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:35.931109 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:35.931056    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8" podStartSLOduration=1.5245572520000001 podStartE2EDuration="1.931038982s" totalImagesPullingTime="406.48173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:35.928977243 +0000 UTC m=+1702.676577158" watchObservedRunningTime="2026-07-14 17:33:35.931038982 +0000 UTC m=+1702.678638859"
Jul 14 17:33:38.345027 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:38.344986    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:33:41.919191 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:41.919146    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:33:46.964720 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:46.964681    2580 generic.go:356] "Generic (PLEG): container finished" podID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerID="f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d" exitCode=0
Jul 14 17:33:46.965158 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:46.964758    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" event={"ID":"546b39ac-2e8a-45f0-a150-c6d14e944c3c","Type":"ContainerDied","Data":"f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d"}
Jul 14 17:33:46.965225 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:46.965209    2580 scope.go:117] "RemoveContainer" containerID="f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d"
Jul 14 17:33:46.966311 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:46.966284    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:33:46.966368 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:46.966354    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:33:46.966405 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:46.966387    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:33:47.972034 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:47.971989    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" event={"ID":"546b39ac-2e8a-45f0-a150-c6d14e944c3c","Type":"ContainerStarted","Data":"a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5"}
Jul 14 17:33:48.317895 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:48.317770    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:48.319107 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:48.319074    2580 patch_prober.go:28] interesting pod/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 14 17:33:48.319249 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:48.319133    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 14 17:33:49.317811 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:49.317770    2580 patch_prober.go:28] interesting pod/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 14 17:33:49.318237 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:49.317830    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 14 17:33:49.902347 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:49.902306    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:50.317716 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:50.317671    2580 patch_prober.go:28] interesting pod/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 14 17:33:50.318015 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:50.317755    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 14 17:33:51.317113 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:51.317074    2580 patch_prober.go:28] interesting pod/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 14 17:33:51.317304 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:51.317132    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 14 17:33:52.316965 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:52.316925    2580 patch_prober.go:28] interesting pod/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 14 17:33:52.317478 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:52.316985    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 14 17:33:53.321404 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:53.321370    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:33:53.322599 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:33:53.322574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:34:50.027592 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.027556    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"]
Jul 14 17:34:50.028125 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.027778    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8" podUID="d41bd93c-41b9-49ec-bf01-1b33262b0a9d" containerName="maas-api" containerID="cri-o://302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c" gracePeriod=30
Jul 14 17:34:50.284078 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.284010    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:34:50.294059 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.294027    2580 generic.go:356] "Generic (PLEG): container finished" podID="d41bd93c-41b9-49ec-bf01-1b33262b0a9d" containerID="302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c" exitCode=0
Jul 14 17:34:50.294211 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.294074    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"
Jul 14 17:34:50.294211 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.294118    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8" event={"ID":"d41bd93c-41b9-49ec-bf01-1b33262b0a9d","Type":"ContainerDied","Data":"302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c"}
Jul 14 17:34:50.294211 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.294172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8" event={"ID":"d41bd93c-41b9-49ec-bf01-1b33262b0a9d","Type":"ContainerDied","Data":"e616e9e3fd83533712369e2f6334d18ae33c96bad756f0a9bae68f2bf95ca585"}
Jul 14 17:34:50.294211 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.294201    2580 scope.go:117] "RemoveContainer" containerID="302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c"
Jul 14 17:34:50.302710 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.302688    2580 scope.go:117] "RemoveContainer" containerID="302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c"
Jul 14 17:34:50.303117 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:34:50.303092    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c\": container with ID starting with 302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c not found: ID does not exist" containerID="302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c"
Jul 14 17:34:50.303195 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.303133    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c"} err="failed to get container status \"302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c\": rpc error: code = NotFound desc = could not find container \"302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c\": container with ID starting with 302956fe69c5cf057b8b5b6b8705793df36711f97d3eed2ba34cf42a5b97d62c not found: ID does not exist"
Jul 14 17:34:50.334350 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.334322    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-openshift-service-ca\") pod \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") "
Jul 14 17:34:50.334350 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.334362    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9h6p\" (UniqueName: \"kubernetes.io/projected/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-kube-api-access-c9h6p\") pod \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") "
Jul 14 17:34:50.334594 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.334406    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-maas-api-tls\") pod \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\" (UID: \"d41bd93c-41b9-49ec-bf01-1b33262b0a9d\") "
Jul 14 17:34:50.334770 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.334745    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d41bd93c-41b9-49ec-bf01-1b33262b0a9d" (UID: "d41bd93c-41b9-49ec-bf01-1b33262b0a9d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:34:50.336689 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.336655    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d41bd93c-41b9-49ec-bf01-1b33262b0a9d" (UID: "d41bd93c-41b9-49ec-bf01-1b33262b0a9d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:34:50.336824 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.336693    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-kube-api-access-c9h6p" (OuterVolumeSpecName: "kube-api-access-c9h6p") pod "d41bd93c-41b9-49ec-bf01-1b33262b0a9d" (UID: "d41bd93c-41b9-49ec-bf01-1b33262b0a9d"). InnerVolumeSpecName "kube-api-access-c9h6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:34:50.435334 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.435295    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-openshift-service-ca\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:34:50.435334 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.435328    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9h6p\" (UniqueName: \"kubernetes.io/projected/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-kube-api-access-c9h6p\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:34:50.435334 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.435338    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d41bd93c-41b9-49ec-bf01-1b33262b0a9d-maas-api-tls\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:34:50.621441 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.621405    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"]
Jul 14 17:34:50.626837 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:50.626808    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30670f0b-b99c9596d-gpgt8"]
Jul 14 17:34:51.881679 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:51.881645    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d41bd93c-41b9-49ec-bf01-1b33262b0a9d" path="/var/lib/kubelet/pods/d41bd93c-41b9-49ec-bf01-1b33262b0a9d/volumes"
Jul 14 17:34:53.142071 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:34:53.142030    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:35:05.692788 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:05.692698    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"]
Jul 14 17:35:05.693216 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:05.692915    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy" containerID="cri-o://a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5" gracePeriod=30
Jul 14 17:35:07.568541 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.568509    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"]
Jul 14 17:35:07.568919 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.568903    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41bd93c-41b9-49ec-bf01-1b33262b0a9d" containerName="maas-api"
Jul 14 17:35:07.568919 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.568914    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41bd93c-41b9-49ec-bf01-1b33262b0a9d" containerName="maas-api"
Jul 14 17:35:07.568996 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.568972    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d41bd93c-41b9-49ec-bf01-1b33262b0a9d" containerName="maas-api"
Jul 14 17:35:07.572632 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.572611    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:07.577409 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.577386    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-74fcaf1a-serving-cert\""
Jul 14 17:35:07.593370 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.593319    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"]
Jul 14 17:35:07.694555 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.694520    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5236b7f7-c763-47f3-b7d0-b36919a76415-openshift-service-ca\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:07.694555 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.694561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:07.694825 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.694698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mzdd\" (UniqueName: \"kubernetes.io/projected/5236b7f7-c763-47f3-b7d0-b36919a76415-kube-api-access-4mzdd\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:07.796050 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.796011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mzdd\" (UniqueName: \"kubernetes.io/projected/5236b7f7-c763-47f3-b7d0-b36919a76415-kube-api-access-4mzdd\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:07.796244 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.796083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5236b7f7-c763-47f3-b7d0-b36919a76415-openshift-service-ca\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:07.796244 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.796111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:07.796244 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:35:07.796228    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-serving-cert: secret "maas-api-e2e-derive-74fcaf1a-serving-cert" not found
Jul 14 17:35:07.796426 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:35:07.796299    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls podName:5236b7f7-c763-47f3-b7d0-b36919a76415 nodeName:}" failed. No retries permitted until 2026-07-14 17:35:08.296283473 +0000 UTC m=+1795.043883328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls") pod "maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29" (UID: "5236b7f7-c763-47f3-b7d0-b36919a76415") : secret "maas-api-e2e-derive-74fcaf1a-serving-cert" not found
Jul 14 17:35:07.796753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.796729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5236b7f7-c763-47f3-b7d0-b36919a76415-openshift-service-ca\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:07.810903 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:07.810864    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mzdd\" (UniqueName: \"kubernetes.io/projected/5236b7f7-c763-47f3-b7d0-b36919a76415-kube-api-access-4mzdd\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:08.302546 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:08.302515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:08.302730 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:35:08.302631    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-serving-cert: secret "maas-api-e2e-derive-74fcaf1a-serving-cert" not found
Jul 14 17:35:08.302730 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:35:08.302690    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls podName:5236b7f7-c763-47f3-b7d0-b36919a76415 nodeName:}" failed. No retries permitted until 2026-07-14 17:35:09.302676287 +0000 UTC m=+1796.050276143 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls") pod "maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29" (UID: "5236b7f7-c763-47f3-b7d0-b36919a76415") : secret "maas-api-e2e-derive-74fcaf1a-serving-cert" not found
Jul 14 17:35:08.324461 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:08.324420    2580 patch_prober.go:28] interesting pod/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 14 17:35:08.324631 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:08.324481    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 17:35:09.312694 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:09.312654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:09.315342 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:09.315318    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls\") pod \"maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:09.383147 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:09.383112    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:09.523438 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:09.523410    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"]
Jul 14 17:35:09.526499 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:35:09.526473    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5236b7f7_c763_47f3_b7d0_b36919a76415.slice/crio-19524a783fd8eb7922283bf067ce68237d5f0c0e31bda510aec90730463cb6e9 WatchSource:0}: Error finding container 19524a783fd8eb7922283bf067ce68237d5f0c0e31bda510aec90730463cb6e9: Status 404 returned error can't find the container with id 19524a783fd8eb7922283bf067ce68237d5f0c0e31bda510aec90730463cb6e9
Jul 14 17:35:09.553022 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:09.552994    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:35:10.398228 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:10.398194    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29" event={"ID":"5236b7f7-c763-47f3-b7d0-b36919a76415","Type":"ContainerStarted","Data":"9aae0d3a16b6b0670144804b0edcf9aa715043c07ff59f182fd78271ab754dd2"}
Jul 14 17:35:10.398718 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:10.398258    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29" event={"ID":"5236b7f7-c763-47f3-b7d0-b36919a76415","Type":"ContainerStarted","Data":"19524a783fd8eb7922283bf067ce68237d5f0c0e31bda510aec90730463cb6e9"}
Jul 14 17:35:10.398718 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:10.398308    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:35:10.426262 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:10.426211    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29" podStartSLOduration=3.035632543 podStartE2EDuration="3.426195291s" totalImagesPullingTime="390.562748ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:35:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:35:10.424445371 +0000 UTC m=+1797.172045249" watchObservedRunningTime="2026-07-14 17:35:10.426195291 +0000 UTC m=+1797.173795167"
Jul 14 17:35:10.941734 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:10.941703    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:35:11.029544 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029495    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istiod-ca-cert\") pod \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") "
Jul 14 17:35:11.029544 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029552    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-envoy\") pod \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") "
Jul 14 17:35:11.029784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029586    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-socket\") pod \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") "
Jul 14 17:35:11.029784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029614    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-credential-socket\") pod \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") "
Jul 14 17:35:11.029784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029639    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-token\") pod \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") "
Jul 14 17:35:11.029784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029671    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-certs\") pod \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") "
Jul 14 17:35:11.029784 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029725    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4ckv\" (UniqueName: \"kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-kube-api-access-n4ckv\") pod \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") "
Jul 14 17:35:11.030042 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029794    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-podinfo\") pod \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") "
Jul 14 17:35:11.030042 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029827    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-data\") pod \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\" (UID: \"546b39ac-2e8a-45f0-a150-c6d14e944c3c\") "
Jul 14 17:35:11.030042 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029931    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "546b39ac-2e8a-45f0-a150-c6d14e944c3c" (UID: "546b39ac-2e8a-45f0-a150-c6d14e944c3c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:35:11.030042 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029939    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "546b39ac-2e8a-45f0-a150-c6d14e944c3c" (UID: "546b39ac-2e8a-45f0-a150-c6d14e944c3c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:35:11.030042 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.029945    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "546b39ac-2e8a-45f0-a150-c6d14e944c3c" (UID: "546b39ac-2e8a-45f0-a150-c6d14e944c3c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:35:11.030280 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.030202    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istiod-ca-cert\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:35:11.030280 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.030258    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "546b39ac-2e8a-45f0-a150-c6d14e944c3c" (UID: "546b39ac-2e8a-45f0-a150-c6d14e944c3c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:35:11.030375 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.030300    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:35:11.030447 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.030432    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-credential-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:35:11.031317 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.031289    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-data" (OuterVolumeSpecName: "istio-data") pod "546b39ac-2e8a-45f0-a150-c6d14e944c3c" (UID: "546b39ac-2e8a-45f0-a150-c6d14e944c3c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:35:11.032153 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.032109    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-token" (OuterVolumeSpecName: "istio-token") pod "546b39ac-2e8a-45f0-a150-c6d14e944c3c" (UID: "546b39ac-2e8a-45f0-a150-c6d14e944c3c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:35:11.032153 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.032130    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "546b39ac-2e8a-45f0-a150-c6d14e944c3c" (UID: "546b39ac-2e8a-45f0-a150-c6d14e944c3c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:35:11.032400 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.032383    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "546b39ac-2e8a-45f0-a150-c6d14e944c3c" (UID: "546b39ac-2e8a-45f0-a150-c6d14e944c3c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 17:35:11.032611 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.032589    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-kube-api-access-n4ckv" (OuterVolumeSpecName: "kube-api-access-n4ckv") pod "546b39ac-2e8a-45f0-a150-c6d14e944c3c" (UID: "546b39ac-2e8a-45f0-a150-c6d14e944c3c"). InnerVolumeSpecName "kube-api-access-n4ckv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:35:11.130928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.130895    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-podinfo\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:35:11.130928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.130922    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-data\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:35:11.130928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.130931    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-envoy\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:35:11.130928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.130940    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-istio-token\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:35:11.131194 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.130947    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/546b39ac-2e8a-45f0-a150-c6d14e944c3c-workload-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:35:11.131194 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.130956    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4ckv\" (UniqueName: \"kubernetes.io/projected/546b39ac-2e8a-45f0-a150-c6d14e944c3c-kube-api-access-n4ckv\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:35:11.404611 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.404517    2580 generic.go:356] "Generic (PLEG): container finished" podID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerID="a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5" exitCode=0
Jul 14 17:35:11.404611 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.404584    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"
Jul 14 17:35:11.404611 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.404601    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" event={"ID":"546b39ac-2e8a-45f0-a150-c6d14e944c3c","Type":"ContainerDied","Data":"a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5"}
Jul 14 17:35:11.405154 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.404681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk" event={"ID":"546b39ac-2e8a-45f0-a150-c6d14e944c3c","Type":"ContainerDied","Data":"324650ca0f974a44271ed4dab72eaac28c17d68f5fb8b3310b379a4f830cf9bb"}
Jul 14 17:35:11.405154 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.404706    2580 scope.go:117] "RemoveContainer" containerID="a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5"
Jul 14 17:35:11.416250 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.416051    2580 scope.go:117] "RemoveContainer" containerID="f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d"
Jul 14 17:35:11.424829 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.424811    2580 scope.go:117] "RemoveContainer" containerID="a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5"
Jul 14 17:35:11.425125 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:35:11.425105    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5\": container with ID starting with a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5 not found: ID does not exist" containerID="a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5"
Jul 14 17:35:11.425178 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.425133    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5"} err="failed to get container status \"a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5\": rpc error: code = NotFound desc = could not find container \"a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5\": container with ID starting with a3effb52cab497d12f80d4c8f98feb4d4785b15a6f7d5a53372ea0b9022392f5 not found: ID does not exist"
Jul 14 17:35:11.425178 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.425149    2580 scope.go:117] "RemoveContainer" containerID="f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d"
Jul 14 17:35:11.425366 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:35:11.425343    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d\": container with ID starting with f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d not found: ID does not exist" containerID="f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d"
Jul 14 17:35:11.425432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.425375    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d"} err="failed to get container status \"f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d\": rpc error: code = NotFound desc = could not find container \"f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d\": container with ID starting with f3900ba3cca3a9e1aebe7070b56a15dd3696ca0dbbe111bc04acfb562fdddb2d not found: ID does not exist"
Jul 14 17:35:11.430399 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.430375    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"]
Jul 14 17:35:11.435879 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.435833    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-30670f0b-openshift-default-65d85dc667-gt4sk"]
Jul 14 17:35:11.882272 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:11.882239    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" path="/var/lib/kubelet/pods/546b39ac-2e8a-45f0-a150-c6d14e944c3c/volumes"
Jul 14 17:35:13.817311 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:13.817283    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:35:13.819283 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:13.819260    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:35:16.410982 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:35:16.410953    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:36:22.524376 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.524341    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"]
Jul 14 17:36:22.524902 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.524612    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29" podUID="5236b7f7-c763-47f3-b7d0-b36919a76415" containerName="maas-api" containerID="cri-o://9aae0d3a16b6b0670144804b0edcf9aa715043c07ff59f182fd78271ab754dd2" gracePeriod=30
Jul 14 17:36:22.757379 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.757342    2580 generic.go:356] "Generic (PLEG): container finished" podID="5236b7f7-c763-47f3-b7d0-b36919a76415" containerID="9aae0d3a16b6b0670144804b0edcf9aa715043c07ff59f182fd78271ab754dd2" exitCode=0
Jul 14 17:36:22.757562 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.757413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29" event={"ID":"5236b7f7-c763-47f3-b7d0-b36919a76415","Type":"ContainerDied","Data":"9aae0d3a16b6b0670144804b0edcf9aa715043c07ff59f182fd78271ab754dd2"}
Jul 14 17:36:22.783445 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.783375    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:36:22.833662 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.833631    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5236b7f7-c763-47f3-b7d0-b36919a76415-openshift-service-ca\") pod \"5236b7f7-c763-47f3-b7d0-b36919a76415\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") "
Jul 14 17:36:22.833858 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.833703    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mzdd\" (UniqueName: \"kubernetes.io/projected/5236b7f7-c763-47f3-b7d0-b36919a76415-kube-api-access-4mzdd\") pod \"5236b7f7-c763-47f3-b7d0-b36919a76415\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") "
Jul 14 17:36:22.833858 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.833746    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls\") pod \"5236b7f7-c763-47f3-b7d0-b36919a76415\" (UID: \"5236b7f7-c763-47f3-b7d0-b36919a76415\") "
Jul 14 17:36:22.834053 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.834030    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5236b7f7-c763-47f3-b7d0-b36919a76415-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5236b7f7-c763-47f3-b7d0-b36919a76415" (UID: "5236b7f7-c763-47f3-b7d0-b36919a76415"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:36:22.836089 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.836060    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5236b7f7-c763-47f3-b7d0-b36919a76415-kube-api-access-4mzdd" (OuterVolumeSpecName: "kube-api-access-4mzdd") pod "5236b7f7-c763-47f3-b7d0-b36919a76415" (UID: "5236b7f7-c763-47f3-b7d0-b36919a76415"). InnerVolumeSpecName "kube-api-access-4mzdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:36:22.836089 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.836063    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5236b7f7-c763-47f3-b7d0-b36919a76415" (UID: "5236b7f7-c763-47f3-b7d0-b36919a76415"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:36:22.934414 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.934379    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5236b7f7-c763-47f3-b7d0-b36919a76415-maas-api-tls\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:36:22.934414 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.934409    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5236b7f7-c763-47f3-b7d0-b36919a76415-openshift-service-ca\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:36:22.934414 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:22.934420    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mzdd\" (UniqueName: \"kubernetes.io/projected/5236b7f7-c763-47f3-b7d0-b36919a76415-kube-api-access-4mzdd\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:36:23.764519 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:23.764484    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29" event={"ID":"5236b7f7-c763-47f3-b7d0-b36919a76415","Type":"ContainerDied","Data":"19524a783fd8eb7922283bf067ce68237d5f0c0e31bda510aec90730463cb6e9"}
Jul 14 17:36:23.764519 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:23.764502    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"
Jul 14 17:36:23.765024 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:23.764553    2580 scope.go:117] "RemoveContainer" containerID="9aae0d3a16b6b0670144804b0edcf9aa715043c07ff59f182fd78271ab754dd2"
Jul 14 17:36:23.793995 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:23.793962    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"]
Jul 14 17:36:23.799544 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:23.799521    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-74fcaf1a-6fbcb495fb-v9z29"]
Jul 14 17:36:23.882876 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:23.882826    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5236b7f7-c763-47f3-b7d0-b36919a76415" path="/var/lib/kubelet/pods/5236b7f7-c763-47f3-b7d0-b36919a76415/volumes"
Jul 14 17:36:25.539766 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:36:25.539727    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:38:08.060750 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:08.060662    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:38:16.097262 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:16.097225    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:38:22.734668 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.734634    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"]
Jul 14 17:38:22.735065 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.735035    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy"
Jul 14 17:38:22.735065 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.735046    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy"
Jul 14 17:38:22.735065 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.735058    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy"
Jul 14 17:38:22.735065 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.735063    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy"
Jul 14 17:38:22.735564 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.735070    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5236b7f7-c763-47f3-b7d0-b36919a76415" containerName="maas-api"
Jul 14 17:38:22.735564 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.735076    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5236b7f7-c763-47f3-b7d0-b36919a76415" containerName="maas-api"
Jul 14 17:38:22.735564 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.735131    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy"
Jul 14 17:38:22.735564 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.735141    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="546b39ac-2e8a-45f0-a150-c6d14e944c3c" containerName="istio-proxy"
Jul 14 17:38:22.735564 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.735147    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5236b7f7-c763-47f3-b7d0-b36919a76415" containerName="maas-api"
Jul 14 17:38:22.738068 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.738051    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.740991 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.740971    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-763702f3-openshift-default-dockercfg-8h6sz\""
Jul 14 17:38:22.769037 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.769005    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"]
Jul 14 17:38:22.776438 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.776402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f54457d-73f0-457f-b72d-3fb97c223037-istio-podinfo\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.776602 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.776451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7xzr\" (UniqueName: \"kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-kube-api-access-d7xzr\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.776602 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.776481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-data\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.776602 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.776523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-socket\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.776743 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.776603    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-credential-socket\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.776743 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.776627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-certs\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.776743 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.776662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-envoy\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.776862 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.776757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f54457d-73f0-457f-b72d-3fb97c223037-istiod-ca-cert\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.776862 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.776797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-istio-token\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878016 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.877968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-envoy\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878222 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f54457d-73f0-457f-b72d-3fb97c223037-istiod-ca-cert\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878222 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-istio-token\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878222 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f54457d-73f0-457f-b72d-3fb97c223037-istio-podinfo\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878222 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7xzr\" (UniqueName: \"kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-kube-api-access-d7xzr\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878222 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-data\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878222 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-socket\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878222 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-credential-socket\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878702 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-certs\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878702 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-data\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878702 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-socket\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878702 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-credential-socket\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878915 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-certs\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.878915 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.878887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f54457d-73f0-457f-b72d-3fb97c223037-istiod-ca-cert\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.880695 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.880674    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-envoy\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.880837 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.880819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f54457d-73f0-457f-b72d-3fb97c223037-istio-podinfo\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.893294 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.893266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-istio-token\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:22.893404 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:22.893364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7xzr\" (UniqueName: \"kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-kube-api-access-d7xzr\") pod \"e2e-mt-763702f3-openshift-default-64c495bbff-fbs24\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") " pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:23.048744 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:23.048644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:23.213673 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:38:23.213641    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f54457d_73f0_457f_b72d_3fb97c223037.slice/crio-c1eaa48ee93f5c6bd470b6eb9a8664446bf3207be1a65c9d79de7101aba78ef6 WatchSource:0}: Error finding container c1eaa48ee93f5c6bd470b6eb9a8664446bf3207be1a65c9d79de7101aba78ef6: Status 404 returned error can't find the container with id c1eaa48ee93f5c6bd470b6eb9a8664446bf3207be1a65c9d79de7101aba78ef6
Jul 14 17:38:23.215814 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:23.215777    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:38:23.215928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:23.215869    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:38:23.215928 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:23.215903    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:38:23.218976 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:23.218953    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"]
Jul 14 17:38:23.331546 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:23.331440    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" event={"ID":"7f54457d-73f0-457f-b72d-3fb97c223037","Type":"ContainerStarted","Data":"6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4"}
Jul 14 17:38:23.331546 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:23.331502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" event={"ID":"7f54457d-73f0-457f-b72d-3fb97c223037","Type":"ContainerStarted","Data":"c1eaa48ee93f5c6bd470b6eb9a8664446bf3207be1a65c9d79de7101aba78ef6"}
Jul 14 17:38:23.363050 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:23.362999    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" podStartSLOduration=1.362982347 podStartE2EDuration="1.362982347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:38:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:38:23.362254574 +0000 UTC m=+1990.109854473" watchObservedRunningTime="2026-07-14 17:38:23.362982347 +0000 UTC m=+1990.110582223"
Jul 14 17:38:24.049082 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:24.049054    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:24.054320 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:24.054293    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:24.338981 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:24.338893    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:24.340024 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:24.340003    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:25.248450 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.248417    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:38:25.640814 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.640729    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"]
Jul 14 17:38:25.644175 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.644151    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:25.649410 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.649387    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-763702f3-serving-cert\""
Jul 14 17:38:25.664042 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.664011    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"]
Jul 14 17:38:25.706622 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.706594    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5lhb\" (UniqueName: \"kubernetes.io/projected/79097b2b-f004-4b9f-9bcd-b6eb19e80423-kube-api-access-c5lhb\") pod \"maas-api-e2e-mt-763702f3-55996bdd57-d2dfq\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:25.706819 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.706651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79097b2b-f004-4b9f-9bcd-b6eb19e80423-maas-api-tls\") pod \"maas-api-e2e-mt-763702f3-55996bdd57-d2dfq\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:25.706819 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.706722    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79097b2b-f004-4b9f-9bcd-b6eb19e80423-openshift-service-ca\") pod \"maas-api-e2e-mt-763702f3-55996bdd57-d2dfq\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:25.807705 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.807672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79097b2b-f004-4b9f-9bcd-b6eb19e80423-maas-api-tls\") pod \"maas-api-e2e-mt-763702f3-55996bdd57-d2dfq\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:25.807930 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.807731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79097b2b-f004-4b9f-9bcd-b6eb19e80423-openshift-service-ca\") pod \"maas-api-e2e-mt-763702f3-55996bdd57-d2dfq\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:25.807930 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.807824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5lhb\" (UniqueName: \"kubernetes.io/projected/79097b2b-f004-4b9f-9bcd-b6eb19e80423-kube-api-access-c5lhb\") pod \"maas-api-e2e-mt-763702f3-55996bdd57-d2dfq\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:25.807930 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:38:25.807827    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-serving-cert: secret "maas-api-e2e-mt-763702f3-serving-cert" not found
Jul 14 17:38:25.808073 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:38:25.807966    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79097b2b-f004-4b9f-9bcd-b6eb19e80423-maas-api-tls podName:79097b2b-f004-4b9f-9bcd-b6eb19e80423 nodeName:}" failed. No retries permitted until 2026-07-14 17:38:26.307943154 +0000 UTC m=+1993.055543023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79097b2b-f004-4b9f-9bcd-b6eb19e80423-maas-api-tls") pod "maas-api-e2e-mt-763702f3-55996bdd57-d2dfq" (UID: "79097b2b-f004-4b9f-9bcd-b6eb19e80423") : secret "maas-api-e2e-mt-763702f3-serving-cert" not found
Jul 14 17:38:25.808398 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.808376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79097b2b-f004-4b9f-9bcd-b6eb19e80423-openshift-service-ca\") pod \"maas-api-e2e-mt-763702f3-55996bdd57-d2dfq\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:25.821625 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:25.821595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5lhb\" (UniqueName: \"kubernetes.io/projected/79097b2b-f004-4b9f-9bcd-b6eb19e80423-kube-api-access-c5lhb\") pod \"maas-api-e2e-mt-763702f3-55996bdd57-d2dfq\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:26.313438 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:26.313391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79097b2b-f004-4b9f-9bcd-b6eb19e80423-maas-api-tls\") pod \"maas-api-e2e-mt-763702f3-55996bdd57-d2dfq\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:26.316057 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:26.316030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79097b2b-f004-4b9f-9bcd-b6eb19e80423-maas-api-tls\") pod \"maas-api-e2e-mt-763702f3-55996bdd57-d2dfq\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:26.555204 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:26.555164    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:26.710462 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:26.710440    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"]
Jul 14 17:38:26.712681 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:38:26.712651    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79097b2b_f004_4b9f_9bcd_b6eb19e80423.slice/crio-8c67d3de2399658425dbbabee5dff73e4bf10ed741571079cbe7ca5db279d8c4 WatchSource:0}: Error finding container 8c67d3de2399658425dbbabee5dff73e4bf10ed741571079cbe7ca5db279d8c4: Status 404 returned error can't find the container with id 8c67d3de2399658425dbbabee5dff73e4bf10ed741571079cbe7ca5db279d8c4
Jul 14 17:38:26.714348 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:26.714327    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 17:38:27.354343 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:27.354304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq" event={"ID":"79097b2b-f004-4b9f-9bcd-b6eb19e80423","Type":"ContainerStarted","Data":"8c67d3de2399658425dbbabee5dff73e4bf10ed741571079cbe7ca5db279d8c4"}
Jul 14 17:38:28.360551 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:28.360513    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq" event={"ID":"79097b2b-f004-4b9f-9bcd-b6eb19e80423","Type":"ContainerStarted","Data":"5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca"}
Jul 14 17:38:28.361042 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:28.360590    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:28.387091 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:28.387042    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq" podStartSLOduration=2.66585814 podStartE2EDuration="3.387025216s" totalImagesPullingTime="721.167076ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:38:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:38:28.384070126 +0000 UTC m=+1995.131670002" watchObservedRunningTime="2026-07-14 17:38:28.387025216 +0000 UTC m=+1995.134625093"
Jul 14 17:38:34.383064 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:34.383025    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:35.649012 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:35.648971    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:38:39.340748 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:39.340653    2580 patch_prober.go:28] interesting pod/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 14 17:38:39.340748 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:39.340723    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 14 17:38:39.422955 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:39.422916    2580 generic.go:356] "Generic (PLEG): container finished" podID="7f54457d-73f0-457f-b72d-3fb97c223037" containerID="6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4" exitCode=0
Jul 14 17:38:39.423129 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:39.422995    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" event={"ID":"7f54457d-73f0-457f-b72d-3fb97c223037","Type":"ContainerDied","Data":"6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4"}
Jul 14 17:38:39.423539 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:39.423521    2580 scope.go:117] "RemoveContainer" containerID="6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4"
Jul 14 17:38:39.424659 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:39.424623    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:38:39.424735 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:39.424717    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:38:39.424782 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:39.424763    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:38:40.430481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:40.430448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" event={"ID":"7f54457d-73f0-457f-b72d-3fb97c223037","Type":"ContainerStarted","Data":"db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376"}
Jul 14 17:38:41.049625 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:41.049591    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:41.049868 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:41.049817    2580 patch_prober.go:28] interesting pod/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 14 17:38:41.049925 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:41.049899    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 14 17:38:42.050028 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:42.049981    2580 patch_prober.go:28] interesting pod/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 14 17:38:42.050507 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:42.050061    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 14 17:38:43.049743 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:43.049707    2580 patch_prober.go:28] interesting pod/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 14 17:38:43.049932 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:43.049763    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 14 17:38:44.049523 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:44.049484    2580 patch_prober.go:28] interesting pod/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 14 17:38:44.049987 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:44.049553    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 14 17:38:45.053376 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:45.053345    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:45.053789 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:45.053692    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:45.054300 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:45.054284    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:38:48.977983 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:48.977946    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"]
Jul 14 17:38:48.978372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:48.978213    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq" podUID="79097b2b-f004-4b9f-9bcd-b6eb19e80423" containerName="maas-api" containerID="cri-o://5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca" gracePeriod=30
Jul 14 17:38:49.235160 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.235091    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:49.346947 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.346904    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5lhb\" (UniqueName: \"kubernetes.io/projected/79097b2b-f004-4b9f-9bcd-b6eb19e80423-kube-api-access-c5lhb\") pod \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") "
Jul 14 17:38:49.347099 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.346978    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79097b2b-f004-4b9f-9bcd-b6eb19e80423-maas-api-tls\") pod \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") "
Jul 14 17:38:49.347099 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.347005    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79097b2b-f004-4b9f-9bcd-b6eb19e80423-openshift-service-ca\") pod \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\" (UID: \"79097b2b-f004-4b9f-9bcd-b6eb19e80423\") "
Jul 14 17:38:49.347383 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.347354    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79097b2b-f004-4b9f-9bcd-b6eb19e80423-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79097b2b-f004-4b9f-9bcd-b6eb19e80423" (UID: "79097b2b-f004-4b9f-9bcd-b6eb19e80423"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:38:49.349242 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.349213    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79097b2b-f004-4b9f-9bcd-b6eb19e80423-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79097b2b-f004-4b9f-9bcd-b6eb19e80423" (UID: "79097b2b-f004-4b9f-9bcd-b6eb19e80423"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:38:49.349353 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.349299    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79097b2b-f004-4b9f-9bcd-b6eb19e80423-kube-api-access-c5lhb" (OuterVolumeSpecName: "kube-api-access-c5lhb") pod "79097b2b-f004-4b9f-9bcd-b6eb19e80423" (UID: "79097b2b-f004-4b9f-9bcd-b6eb19e80423"). InnerVolumeSpecName "kube-api-access-c5lhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:38:49.448631 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.448594    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5lhb\" (UniqueName: \"kubernetes.io/projected/79097b2b-f004-4b9f-9bcd-b6eb19e80423-kube-api-access-c5lhb\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:38:49.448631 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.448625    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79097b2b-f004-4b9f-9bcd-b6eb19e80423-maas-api-tls\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:38:49.448631 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.448635    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79097b2b-f004-4b9f-9bcd-b6eb19e80423-openshift-service-ca\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:38:49.476059 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.476023    2580 generic.go:356] "Generic (PLEG): container finished" podID="79097b2b-f004-4b9f-9bcd-b6eb19e80423" containerID="5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca" exitCode=0
Jul 14 17:38:49.476244 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.476066    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"
Jul 14 17:38:49.476244 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.476105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq" event={"ID":"79097b2b-f004-4b9f-9bcd-b6eb19e80423","Type":"ContainerDied","Data":"5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca"}
Jul 14 17:38:49.476244 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.476168    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq" event={"ID":"79097b2b-f004-4b9f-9bcd-b6eb19e80423","Type":"ContainerDied","Data":"8c67d3de2399658425dbbabee5dff73e4bf10ed741571079cbe7ca5db279d8c4"}
Jul 14 17:38:49.476244 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.476198    2580 scope.go:117] "RemoveContainer" containerID="5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca"
Jul 14 17:38:49.485511 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.485491    2580 scope.go:117] "RemoveContainer" containerID="5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca"
Jul 14 17:38:49.485809 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:38:49.485790    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca\": container with ID starting with 5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca not found: ID does not exist" containerID="5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca"
Jul 14 17:38:49.485899 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.485819    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca"} err="failed to get container status \"5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca\": rpc error: code = NotFound desc = could not find container \"5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca\": container with ID starting with 5b2e6ab26581b04d30b396ca18cc666ad57ac045882df5a793685226c93e1fca not found: ID does not exist"
Jul 14 17:38:49.507758 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.507727    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"]
Jul 14 17:38:49.513464 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.513438    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-763702f3-55996bdd57-d2dfq"]
Jul 14 17:38:49.882364 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:49.882278    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79097b2b-f004-4b9f-9bcd-b6eb19e80423" path="/var/lib/kubelet/pods/79097b2b-f004-4b9f-9bcd-b6eb19e80423/volumes"
Jul 14 17:38:51.945747 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:38:51.945709    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:39:04.271515 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:04.271471    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"]
Jul 14 17:39:04.272015 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:04.271731    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy" containerID="cri-o://db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376" gracePeriod=30
Jul 14 17:39:09.529359 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.529332    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:39:09.576810 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.576726    2580 generic.go:356] "Generic (PLEG): container finished" podID="7f54457d-73f0-457f-b72d-3fb97c223037" containerID="db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376" exitCode=0
Jul 14 17:39:09.576980 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.576828    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"
Jul 14 17:39:09.576980 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.576879    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" event={"ID":"7f54457d-73f0-457f-b72d-3fb97c223037","Type":"ContainerDied","Data":"db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376"}
Jul 14 17:39:09.576980 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.576932    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24" event={"ID":"7f54457d-73f0-457f-b72d-3fb97c223037","Type":"ContainerDied","Data":"c1eaa48ee93f5c6bd470b6eb9a8664446bf3207be1a65c9d79de7101aba78ef6"}
Jul 14 17:39:09.576980 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.576954    2580 scope.go:117] "RemoveContainer" containerID="db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376"
Jul 14 17:39:09.586410 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.586386    2580 scope.go:117] "RemoveContainer" containerID="6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4"
Jul 14 17:39:09.595127 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.595104    2580 scope.go:117] "RemoveContainer" containerID="db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376"
Jul 14 17:39:09.595428 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:39:09.595405    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376\": container with ID starting with db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376 not found: ID does not exist" containerID="db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376"
Jul 14 17:39:09.595507 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.595443    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376"} err="failed to get container status \"db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376\": rpc error: code = NotFound desc = could not find container \"db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376\": container with ID starting with db04b58dbdb032d8f9778c77a7cbbcb0144b76f571edfcc8bdeafaf993ad7376 not found: ID does not exist"
Jul 14 17:39:09.595507 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.595470    2580 scope.go:117] "RemoveContainer" containerID="6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4"
Jul 14 17:39:09.595801 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:39:09.595783    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4\": container with ID starting with 6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4 not found: ID does not exist" containerID="6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4"
Jul 14 17:39:09.595867 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.595808    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4"} err="failed to get container status \"6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4\": rpc error: code = NotFound desc = could not find container \"6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4\": container with ID starting with 6b252d78cff54ae68b99b9a3de344992264cd9244e2f5956dbbdd661a9b2b7e4 not found: ID does not exist"
Jul 14 17:39:09.627768 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.627743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-data\") pod \"7f54457d-73f0-457f-b72d-3fb97c223037\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") "
Jul 14 17:39:09.627895 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.627779    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f54457d-73f0-457f-b72d-3fb97c223037-istiod-ca-cert\") pod \"7f54457d-73f0-457f-b72d-3fb97c223037\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") "
Jul 14 17:39:09.627895 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.627810    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-credential-socket\") pod \"7f54457d-73f0-457f-b72d-3fb97c223037\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") "
Jul 14 17:39:09.627895 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.627835    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-istio-token\") pod \"7f54457d-73f0-457f-b72d-3fb97c223037\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") "
Jul 14 17:39:09.627895 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.627891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-socket\") pod \"7f54457d-73f0-457f-b72d-3fb97c223037\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") "
Jul 14 17:39:09.628101 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.627913    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-envoy\") pod \"7f54457d-73f0-457f-b72d-3fb97c223037\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") "
Jul 14 17:39:09.628101 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.627969    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-certs\") pod \"7f54457d-73f0-457f-b72d-3fb97c223037\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") "
Jul 14 17:39:09.628101 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.628000    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f54457d-73f0-457f-b72d-3fb97c223037-istio-podinfo\") pod \"7f54457d-73f0-457f-b72d-3fb97c223037\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") "
Jul 14 17:39:09.628101 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.628054    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7xzr\" (UniqueName: \"kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-kube-api-access-d7xzr\") pod \"7f54457d-73f0-457f-b72d-3fb97c223037\" (UID: \"7f54457d-73f0-457f-b72d-3fb97c223037\") "
Jul 14 17:39:09.628101 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.628063    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f54457d-73f0-457f-b72d-3fb97c223037" (UID: "7f54457d-73f0-457f-b72d-3fb97c223037"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:39:09.628346 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.628233    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f54457d-73f0-457f-b72d-3fb97c223037-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f54457d-73f0-457f-b72d-3fb97c223037" (UID: "7f54457d-73f0-457f-b72d-3fb97c223037"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:39:09.628346 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.628236    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f54457d-73f0-457f-b72d-3fb97c223037" (UID: "7f54457d-73f0-457f-b72d-3fb97c223037"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:39:09.628469 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.628380    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f54457d-73f0-457f-b72d-3fb97c223037" (UID: "7f54457d-73f0-457f-b72d-3fb97c223037"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:39:09.628469 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.628456    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f54457d-73f0-457f-b72d-3fb97c223037-istiod-ca-cert\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:39:09.628571 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.628478    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-credential-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:39:09.628571 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.628496    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:39:09.629029 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.628999    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f54457d-73f0-457f-b72d-3fb97c223037" (UID: "7f54457d-73f0-457f-b72d-3fb97c223037"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:39:09.630419 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.630392    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f54457d-73f0-457f-b72d-3fb97c223037-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f54457d-73f0-457f-b72d-3fb97c223037" (UID: "7f54457d-73f0-457f-b72d-3fb97c223037"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 17:39:09.630419 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.630406    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f54457d-73f0-457f-b72d-3fb97c223037" (UID: "7f54457d-73f0-457f-b72d-3fb97c223037"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:39:09.630550 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.630444    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-kube-api-access-d7xzr" (OuterVolumeSpecName: "kube-api-access-d7xzr") pod "7f54457d-73f0-457f-b72d-3fb97c223037" (UID: "7f54457d-73f0-457f-b72d-3fb97c223037"). InnerVolumeSpecName "kube-api-access-d7xzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:39:09.630550 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.630458    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f54457d-73f0-457f-b72d-3fb97c223037" (UID: "7f54457d-73f0-457f-b72d-3fb97c223037"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:39:09.729954 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.729915    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-istio-token\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:39:09.729954 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.729947    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-workload-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:39:09.729954 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.729958    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-envoy\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:39:09.730204 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.729968    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f54457d-73f0-457f-b72d-3fb97c223037-istio-podinfo\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:39:09.730204 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.729976    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7xzr\" (UniqueName: \"kubernetes.io/projected/7f54457d-73f0-457f-b72d-3fb97c223037-kube-api-access-d7xzr\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:39:09.730204 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.729984    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f54457d-73f0-457f-b72d-3fb97c223037-istio-data\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:39:09.905327 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.905295    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"]
Jul 14 17:39:09.910700 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:09.910674    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-763702f3-openshift-default-64c495bbff-fbs24"]
Jul 14 17:39:11.882522 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:11.882483    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" path="/var/lib/kubelet/pods/7f54457d-73f0-457f-b72d-3fb97c223037/volumes"
Jul 14 17:39:32.018088 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.018052    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"]
Jul 14 17:39:32.018456 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.018445    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79097b2b-f004-4b9f-9bcd-b6eb19e80423" containerName="maas-api"
Jul 14 17:39:32.018456 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.018456    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="79097b2b-f004-4b9f-9bcd-b6eb19e80423" containerName="maas-api"
Jul 14 17:39:32.018534 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.018475    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy"
Jul 14 17:39:32.018534 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.018481    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy"
Jul 14 17:39:32.018595 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.018554    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="79097b2b-f004-4b9f-9bcd-b6eb19e80423" containerName="maas-api"
Jul 14 17:39:32.018595 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.018565    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy"
Jul 14 17:39:32.021813 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.021788    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.027247 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.027226    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9cc7492b-serving-cert\""
Jul 14 17:39:32.041143 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.041115    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"]
Jul 14 17:39:32.145414 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.145374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbe886ce-3ebf-4b4f-91fc-704deca2274e-openshift-service-ca\") pod \"maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.145414 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.145411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnkhb\" (UniqueName: \"kubernetes.io/projected/fbe886ce-3ebf-4b4f-91fc-704deca2274e-kube-api-access-hnkhb\") pod \"maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.145640 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.145433    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbe886ce-3ebf-4b4f-91fc-704deca2274e-maas-api-tls\") pod \"maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.246458 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.246419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbe886ce-3ebf-4b4f-91fc-704deca2274e-openshift-service-ca\") pod \"maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.246458 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.246461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnkhb\" (UniqueName: \"kubernetes.io/projected/fbe886ce-3ebf-4b4f-91fc-704deca2274e-kube-api-access-hnkhb\") pod \"maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.246675 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.246488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbe886ce-3ebf-4b4f-91fc-704deca2274e-maas-api-tls\") pod \"maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.246675 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:39:32.246625    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-serving-cert: secret "maas-api-e2e-mt-9cc7492b-serving-cert" not found
Jul 14 17:39:32.246745 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:39:32.246690    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe886ce-3ebf-4b4f-91fc-704deca2274e-maas-api-tls podName:fbe886ce-3ebf-4b4f-91fc-704deca2274e nodeName:}" failed. No retries permitted until 2026-07-14 17:39:32.746669591 +0000 UTC m=+2059.494269447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fbe886ce-3ebf-4b4f-91fc-704deca2274e-maas-api-tls") pod "maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf" (UID: "fbe886ce-3ebf-4b4f-91fc-704deca2274e") : secret "maas-api-e2e-mt-9cc7492b-serving-cert" not found
Jul 14 17:39:32.247173 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.247151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbe886ce-3ebf-4b4f-91fc-704deca2274e-openshift-service-ca\") pod \"maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.269513 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.269443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnkhb\" (UniqueName: \"kubernetes.io/projected/fbe886ce-3ebf-4b4f-91fc-704deca2274e-kube-api-access-hnkhb\") pod \"maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.751243 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.751204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbe886ce-3ebf-4b4f-91fc-704deca2274e-maas-api-tls\") pod \"maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.753888 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.753856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbe886ce-3ebf-4b4f-91fc-704deca2274e-maas-api-tls\") pod \"maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:32.931236 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:32.931202    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:33.088006 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:33.087972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"]
Jul 14 17:39:33.090228 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:39:33.090201    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe886ce_3ebf_4b4f_91fc_704deca2274e.slice/crio-3477f2c121f6160cfa9806caf4826b542c77e3a2674729ab90f5d9ff3afc8c9f WatchSource:0}: Error finding container 3477f2c121f6160cfa9806caf4826b542c77e3a2674729ab90f5d9ff3afc8c9f: Status 404 returned error can't find the container with id 3477f2c121f6160cfa9806caf4826b542c77e3a2674729ab90f5d9ff3afc8c9f
Jul 14 17:39:33.698383 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:33.698298    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf" event={"ID":"fbe886ce-3ebf-4b4f-91fc-704deca2274e","Type":"ContainerStarted","Data":"ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5"}
Jul 14 17:39:33.698383 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:33.698359    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf" event={"ID":"fbe886ce-3ebf-4b4f-91fc-704deca2274e","Type":"ContainerStarted","Data":"3477f2c121f6160cfa9806caf4826b542c77e3a2674729ab90f5d9ff3afc8c9f"}
Jul 14 17:39:33.698575 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:33.698388    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:33.743294 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:33.743245    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf" podStartSLOduration=2.329026426 podStartE2EDuration="2.743226164s" totalImagesPullingTime="414.199738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:39:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:39:33.742801941 +0000 UTC m=+2060.490401838" watchObservedRunningTime="2026-07-14 17:39:33.743226164 +0000 UTC m=+2060.490826041"
Jul 14 17:39:37.994825 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:37.994790    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:39:39.708521 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:39.708490    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:39:50.076405 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.076369    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"]
Jul 14 17:39:50.076911 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.076763    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy"
Jul 14 17:39:50.076911 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.076776    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy"
Jul 14 17:39:50.076994 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.076943    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f54457d-73f0-457f-b72d-3fb97c223037" containerName="istio-proxy"
Jul 14 17:39:50.079790 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.079771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.086300 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.086282    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9cc7492b\"/\"e2e-lifecycle-model-9cc7492b-kserve-self-signed-certs\""
Jul 14 17:39:50.087548 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.087526    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9cc7492b\"/\"kube-root-ca.crt\""
Jul 14 17:39:50.087548 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.087545    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9cc7492b\"/\"openshift-service-ca.crt\""
Jul 14 17:39:50.093560 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.093540    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9cc7492b\"/\"default-dockercfg-phglx\""
Jul 14 17:39:50.098194 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.098170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb8z9\" (UniqueName: \"kubernetes.io/projected/5946950e-75fb-45de-a608-bd9842cc3df1-kube-api-access-jb8z9\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.098294 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.098219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-tmp-dir\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.098294 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.098241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-model-cache\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.098383 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.098321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-dshm\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.098383 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.098360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5946950e-75fb-45de-a608-bd9842cc3df1-tls-certs\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.098507 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.098484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-home\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.102375 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.102354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"]
Jul 14 17:39:50.198965 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.198925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-dshm\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.198965 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.198967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5946950e-75fb-45de-a608-bd9842cc3df1-tls-certs\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.199228 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.199046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-home\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.199228 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.199076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb8z9\" (UniqueName: \"kubernetes.io/projected/5946950e-75fb-45de-a608-bd9842cc3df1-kube-api-access-jb8z9\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.199228 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.199105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-tmp-dir\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.199228 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.199122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-model-cache\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.199504 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.199482    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-home\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.199572 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.199507    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-model-cache\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.199572 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.199549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-tmp-dir\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.201477 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.201453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-dshm\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.201860 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.201820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5946950e-75fb-45de-a608-bd9842cc3df1-tls-certs\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.217427 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.217401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb8z9\" (UniqueName: \"kubernetes.io/projected/5946950e-75fb-45de-a608-bd9842cc3df1-kube-api-access-jb8z9\") pod \"e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") " pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.388931 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.388813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:39:50.540199 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.540165    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"]
Jul 14 17:39:50.543388 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:39:50.543355    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5946950e_75fb_45de_a608_bd9842cc3df1.slice/crio-ea8fe235750036347402adbacf95a3ae2de2e48e7e2708849ea9ff5d7a499090 WatchSource:0}: Error finding container ea8fe235750036347402adbacf95a3ae2de2e48e7e2708849ea9ff5d7a499090: Status 404 returned error can't find the container with id ea8fe235750036347402adbacf95a3ae2de2e48e7e2708849ea9ff5d7a499090
Jul 14 17:39:50.782404 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.782362    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd" event={"ID":"5946950e-75fb-45de-a608-bd9842cc3df1","Type":"ContainerStarted","Data":"360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5"}
Jul 14 17:39:50.782404 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.782407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd" event={"ID":"5946950e-75fb-45de-a608-bd9842cc3df1","Type":"ContainerStarted","Data":"ea8fe235750036347402adbacf95a3ae2de2e48e7e2708849ea9ff5d7a499090"}
Jul 14 17:39:50.819728 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:50.819675    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd" podStartSLOduration=0.819660233 podStartE2EDuration="819.660233ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:39:50.816156125 +0000 UTC m=+2077.563756002" watchObservedRunningTime="2026-07-14 17:39:50.819660233 +0000 UTC m=+2077.567260103"
Jul 14 17:39:53.660103 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:39:53.660067    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:40:00.390000 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:00.389964    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:40:00.390000 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:00.390017    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:40:00.402794 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:00.402762    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:40:00.842024 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:00.841993    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:40:07.072337 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:07.072300    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:40:13.845957 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:13.845927    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:40:13.850277 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:13.850250    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:40:21.218312 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.218274    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"]
Jul 14 17:40:21.218827 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.218503    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf" podUID="fbe886ce-3ebf-4b4f-91fc-704deca2274e" containerName="maas-api" containerID="cri-o://ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5" gracePeriod=30
Jul 14 17:40:21.467250 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.467220    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:40:21.550026 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.549932    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnkhb\" (UniqueName: \"kubernetes.io/projected/fbe886ce-3ebf-4b4f-91fc-704deca2274e-kube-api-access-hnkhb\") pod \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") "
Jul 14 17:40:21.550026 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.549984    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbe886ce-3ebf-4b4f-91fc-704deca2274e-maas-api-tls\") pod \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") "
Jul 14 17:40:21.550026 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.550025    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbe886ce-3ebf-4b4f-91fc-704deca2274e-openshift-service-ca\") pod \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\" (UID: \"fbe886ce-3ebf-4b4f-91fc-704deca2274e\") "
Jul 14 17:40:21.550406 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.550379    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbe886ce-3ebf-4b4f-91fc-704deca2274e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fbe886ce-3ebf-4b4f-91fc-704deca2274e" (UID: "fbe886ce-3ebf-4b4f-91fc-704deca2274e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:40:21.552267 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.552236    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbe886ce-3ebf-4b4f-91fc-704deca2274e-kube-api-access-hnkhb" (OuterVolumeSpecName: "kube-api-access-hnkhb") pod "fbe886ce-3ebf-4b4f-91fc-704deca2274e" (UID: "fbe886ce-3ebf-4b4f-91fc-704deca2274e"). InnerVolumeSpecName "kube-api-access-hnkhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:40:21.552267 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.552239    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbe886ce-3ebf-4b4f-91fc-704deca2274e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fbe886ce-3ebf-4b4f-91fc-704deca2274e" (UID: "fbe886ce-3ebf-4b4f-91fc-704deca2274e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:40:21.651590 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.651558    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnkhb\" (UniqueName: \"kubernetes.io/projected/fbe886ce-3ebf-4b4f-91fc-704deca2274e-kube-api-access-hnkhb\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:40:21.651590 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.651592    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbe886ce-3ebf-4b4f-91fc-704deca2274e-maas-api-tls\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:40:21.651808 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.651608    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbe886ce-3ebf-4b4f-91fc-704deca2274e-openshift-service-ca\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:40:21.936751 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.936712    2580 generic.go:356] "Generic (PLEG): container finished" podID="fbe886ce-3ebf-4b4f-91fc-704deca2274e" containerID="ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5" exitCode=0
Jul 14 17:40:21.936751 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.936753    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"
Jul 14 17:40:21.936993 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.936793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf" event={"ID":"fbe886ce-3ebf-4b4f-91fc-704deca2274e","Type":"ContainerDied","Data":"ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5"}
Jul 14 17:40:21.936993 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.936872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf" event={"ID":"fbe886ce-3ebf-4b4f-91fc-704deca2274e","Type":"ContainerDied","Data":"3477f2c121f6160cfa9806caf4826b542c77e3a2674729ab90f5d9ff3afc8c9f"}
Jul 14 17:40:21.936993 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.936896    2580 scope.go:117] "RemoveContainer" containerID="ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5"
Jul 14 17:40:21.945525 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.945509    2580 scope.go:117] "RemoveContainer" containerID="ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5"
Jul 14 17:40:21.945770 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:40:21.945751    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5\": container with ID starting with ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5 not found: ID does not exist" containerID="ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5"
Jul 14 17:40:21.945828 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.945778    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5"} err="failed to get container status \"ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5\": rpc error: code = NotFound desc = could not find container \"ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5\": container with ID starting with ba0621118d4533b54d23dd23f4827c9cc7d39cc03821704f404604405a9638b5 not found: ID does not exist"
Jul 14 17:40:21.957714 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.957690    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"]
Jul 14 17:40:21.960647 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:21.960623    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cc7492b-7756d75cd-8knxf"]
Jul 14 17:40:23.882079 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:23.882044    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbe886ce-3ebf-4b4f-91fc-704deca2274e" path="/var/lib/kubelet/pods/fbe886ce-3ebf-4b4f-91fc-704deca2274e/volumes"
Jul 14 17:40:25.462750 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:25.462709    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:40:30.346571 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:30.346536    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"]
Jul 14 17:40:45.368044 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.367974    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd" podUID="5946950e-75fb-45de-a608-bd9842cc3df1" containerName="main" containerID="cri-o://360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5" gracePeriod=45
Jul 14 17:40:45.622836 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.622769    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:40:45.782297 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782261    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-model-cache\") pod \"5946950e-75fb-45de-a608-bd9842cc3df1\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") "
Jul 14 17:40:45.782297 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782304    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-home\") pod \"5946950e-75fb-45de-a608-bd9842cc3df1\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") "
Jul 14 17:40:45.782525 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782365    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-tmp-dir\") pod \"5946950e-75fb-45de-a608-bd9842cc3df1\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") "
Jul 14 17:40:45.782525 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782393    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb8z9\" (UniqueName: \"kubernetes.io/projected/5946950e-75fb-45de-a608-bd9842cc3df1-kube-api-access-jb8z9\") pod \"5946950e-75fb-45de-a608-bd9842cc3df1\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") "
Jul 14 17:40:45.782525 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782469    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5946950e-75fb-45de-a608-bd9842cc3df1-tls-certs\") pod \"5946950e-75fb-45de-a608-bd9842cc3df1\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") "
Jul 14 17:40:45.782525 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782521    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-dshm\") pod \"5946950e-75fb-45de-a608-bd9842cc3df1\" (UID: \"5946950e-75fb-45de-a608-bd9842cc3df1\") "
Jul 14 17:40:45.782666 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782588    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-model-cache" (OuterVolumeSpecName: "model-cache") pod "5946950e-75fb-45de-a608-bd9842cc3df1" (UID: "5946950e-75fb-45de-a608-bd9842cc3df1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:40:45.782720 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782649    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-home" (OuterVolumeSpecName: "home") pod "5946950e-75fb-45de-a608-bd9842cc3df1" (UID: "5946950e-75fb-45de-a608-bd9842cc3df1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:40:45.782720 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782700    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5946950e-75fb-45de-a608-bd9842cc3df1" (UID: "5946950e-75fb-45de-a608-bd9842cc3df1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:40:45.782919 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782896    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-tmp-dir\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:40:45.782919 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782918    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-model-cache\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:40:45.783070 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.782927    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-home\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:40:45.784946 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.784922    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5946950e-75fb-45de-a608-bd9842cc3df1-kube-api-access-jb8z9" (OuterVolumeSpecName: "kube-api-access-jb8z9") pod "5946950e-75fb-45de-a608-bd9842cc3df1" (UID: "5946950e-75fb-45de-a608-bd9842cc3df1"). InnerVolumeSpecName "kube-api-access-jb8z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:40:45.785053 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.784953    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5946950e-75fb-45de-a608-bd9842cc3df1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5946950e-75fb-45de-a608-bd9842cc3df1" (UID: "5946950e-75fb-45de-a608-bd9842cc3df1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:40:45.785269 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.785242    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-dshm" (OuterVolumeSpecName: "dshm") pod "5946950e-75fb-45de-a608-bd9842cc3df1" (UID: "5946950e-75fb-45de-a608-bd9842cc3df1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:40:45.883376 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.883300    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5946950e-75fb-45de-a608-bd9842cc3df1-dshm\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:40:45.883376 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.883325    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb8z9\" (UniqueName: \"kubernetes.io/projected/5946950e-75fb-45de-a608-bd9842cc3df1-kube-api-access-jb8z9\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:40:45.883376 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:45.883335    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5946950e-75fb-45de-a608-bd9842cc3df1-tls-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:40:46.048159 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:46.048122    2580 generic.go:356] "Generic (PLEG): container finished" podID="5946950e-75fb-45de-a608-bd9842cc3df1" containerID="360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5" exitCode=0
Jul 14 17:40:46.048341 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:46.048170    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"
Jul 14 17:40:46.048341 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:46.048187    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd" event={"ID":"5946950e-75fb-45de-a608-bd9842cc3df1","Type":"ContainerDied","Data":"360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5"}
Jul 14 17:40:46.048341 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:46.048226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd" event={"ID":"5946950e-75fb-45de-a608-bd9842cc3df1","Type":"ContainerDied","Data":"ea8fe235750036347402adbacf95a3ae2de2e48e7e2708849ea9ff5d7a499090"}
Jul 14 17:40:46.048341 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:46.048243    2580 scope.go:117] "RemoveContainer" containerID="360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5"
Jul 14 17:40:46.059270 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:46.059246    2580 scope.go:117] "RemoveContainer" containerID="360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5"
Jul 14 17:40:46.059558 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:40:46.059539    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5\": container with ID starting with 360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5 not found: ID does not exist" containerID="360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5"
Jul 14 17:40:46.059614 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:46.059568    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5"} err="failed to get container status \"360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5\": rpc error: code = NotFound desc = could not find container \"360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5\": container with ID starting with 360b9227eb24ec8b51018b954173a761872ec3929ae7dfcefe9708fb8525b9f5 not found: ID does not exist"
Jul 14 17:40:46.091362 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:46.091331    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"]
Jul 14 17:40:46.098912 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:46.098884    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-9cc7492b/e2e-lifecycle-model-9cc7492b-kserve-cd6d5c74c-49fjd"]
Jul 14 17:40:47.881933 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:40:47.881899    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5946950e-75fb-45de-a608-bd9842cc3df1" path="/var/lib/kubelet/pods/5946950e-75fb-45de-a608-bd9842cc3df1/volumes"
Jul 14 17:41:21.644415 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:41:21.644374    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:41:36.641149 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:41:36.641111    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:42:17.797292 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.797257    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"]
Jul 14 17:42:17.797795 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.797619    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5946950e-75fb-45de-a608-bd9842cc3df1" containerName="main"
Jul 14 17:42:17.797795 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.797629    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5946950e-75fb-45de-a608-bd9842cc3df1" containerName="main"
Jul 14 17:42:17.797795 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.797643    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbe886ce-3ebf-4b4f-91fc-704deca2274e" containerName="maas-api"
Jul 14 17:42:17.797795 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.797648    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbe886ce-3ebf-4b4f-91fc-704deca2274e" containerName="maas-api"
Jul 14 17:42:17.797795 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.797714    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbe886ce-3ebf-4b4f-91fc-704deca2274e" containerName="maas-api"
Jul 14 17:42:17.797795 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.797721    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5946950e-75fb-45de-a608-bd9842cc3df1" containerName="main"
Jul 14 17:42:17.800584 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.800568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:17.804714 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.804686    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-d44b45-serving-cert\""
Jul 14 17:42:17.814747 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.814722    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"]
Jul 14 17:42:17.899450 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.899410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzbxp\" (UniqueName: \"kubernetes.io/projected/b60781ff-c287-4616-a45d-a726c08552f3-kube-api-access-rzbxp\") pod \"maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:17.899619 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.899485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b60781ff-c287-4616-a45d-a726c08552f3-maas-api-tls\") pod \"maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:17.899619 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.899548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60781ff-c287-4616-a45d-a726c08552f3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:17.999947 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.999914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60781ff-c287-4616-a45d-a726c08552f3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:18.000128 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:17.999961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzbxp\" (UniqueName: \"kubernetes.io/projected/b60781ff-c287-4616-a45d-a726c08552f3-kube-api-access-rzbxp\") pod \"maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:18.000128 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:18.000005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b60781ff-c287-4616-a45d-a726c08552f3-maas-api-tls\") pod \"maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:18.000128 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:42:18.000125    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-serving-cert: secret "maas-api-e2e-inf-a-d44b45-serving-cert" not found
Jul 14 17:42:18.000265 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:42:18.000196    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b60781ff-c287-4616-a45d-a726c08552f3-maas-api-tls podName:b60781ff-c287-4616-a45d-a726c08552f3 nodeName:}" failed. No retries permitted until 2026-07-14 17:42:18.500175496 +0000 UTC m=+2225.247775560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b60781ff-c287-4616-a45d-a726c08552f3-maas-api-tls") pod "maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9" (UID: "b60781ff-c287-4616-a45d-a726c08552f3") : secret "maas-api-e2e-inf-a-d44b45-serving-cert" not found
Jul 14 17:42:18.000554 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:18.000533    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60781ff-c287-4616-a45d-a726c08552f3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:18.017289 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:18.017260    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzbxp\" (UniqueName: \"kubernetes.io/projected/b60781ff-c287-4616-a45d-a726c08552f3-kube-api-access-rzbxp\") pod \"maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:18.505592 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:18.505553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b60781ff-c287-4616-a45d-a726c08552f3-maas-api-tls\") pod \"maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:18.508361 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:18.508331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b60781ff-c287-4616-a45d-a726c08552f3-maas-api-tls\") pod \"maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:18.711507 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:18.711459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:18.851378 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:18.851356    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"]
Jul 14 17:42:18.853948 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:42:18.853917    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60781ff_c287_4616_a45d_a726c08552f3.slice/crio-4f4d5ec87f0872d446fe6cb6b4c7737bf71456f22c6180812417b280b2835806 WatchSource:0}: Error finding container 4f4d5ec87f0872d446fe6cb6b4c7737bf71456f22c6180812417b280b2835806: Status 404 returned error can't find the container with id 4f4d5ec87f0872d446fe6cb6b4c7737bf71456f22c6180812417b280b2835806
Jul 14 17:42:19.489906 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:19.489866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9" event={"ID":"b60781ff-c287-4616-a45d-a726c08552f3","Type":"ContainerStarted","Data":"77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a"}
Jul 14 17:42:19.490096 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:19.489929    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9" event={"ID":"b60781ff-c287-4616-a45d-a726c08552f3","Type":"ContainerStarted","Data":"4f4d5ec87f0872d446fe6cb6b4c7737bf71456f22c6180812417b280b2835806"}
Jul 14 17:42:19.490096 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:19.489973    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:19.510783 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:19.510689    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9" podStartSLOduration=2.110400106 podStartE2EDuration="2.510673338s" totalImagesPullingTime="400.273232ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:42:19.509748544 +0000 UTC m=+2226.257348452" watchObservedRunningTime="2026-07-14 17:42:19.510673338 +0000 UTC m=+2226.258273193"
Jul 14 17:42:22.036863 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.036804    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:42:22.621399 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.621367    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"]
Jul 14 17:42:22.625071 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.625043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.627866 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.627818    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1b06a1-openshift-default-dockercfg-89klp\""
Jul 14 17:42:22.637773 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.637750    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"]
Jul 14 17:42:22.738977 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.738937    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-credential-socket\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.739139 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.738988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-podinfo\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.739139 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.739024    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-token\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.739139 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.739064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v67hj\" (UniqueName: \"kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-kube-api-access-v67hj\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.739251 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.739152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-socket\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.739251 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.739216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-envoy\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.739313 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.739256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istiod-ca-cert\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.739313 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.739297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-data\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.739379 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.739331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-certs\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.840603 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.840554    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-credential-socket\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.840603 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.840593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-podinfo\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.840603 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.840612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-token\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841017 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.840764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v67hj\" (UniqueName: \"kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-kube-api-access-v67hj\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841017 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.840799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-socket\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841017 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.840829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-envoy\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841017 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.840914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istiod-ca-cert\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841017 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.840972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-data\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841017 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.841014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-certs\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841276 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.841032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-credential-socket\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841276 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.841214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-socket\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841519 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.841496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-data\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841586 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.841541    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-certs\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.841638 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.841600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istiod-ca-cert\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.843290 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.843267    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-podinfo\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.843707 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.843687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-envoy\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.850172 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.850148    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v67hj\" (UniqueName: \"kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-kube-api-access-v67hj\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.850610 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.850592    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-token\") pod \"e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") " pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:22.936011 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:22.935911    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:23.092899 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:23.092863    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"]
Jul 14 17:42:23.094026 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:42:23.093998    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5877ad3_d063_4847_9cb3_fc33c772ee3f.slice/crio-0a6b3c7715cc7339a4df6f392b3e252e53696f947824d42c2fc07eb77d4d0918 WatchSource:0}: Error finding container 0a6b3c7715cc7339a4df6f392b3e252e53696f947824d42c2fc07eb77d4d0918: Status 404 returned error can't find the container with id 0a6b3c7715cc7339a4df6f392b3e252e53696f947824d42c2fc07eb77d4d0918
Jul 14 17:42:23.098900 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:23.098833    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:42:23.099010 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:23.098949    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:42:23.099010 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:23.098992    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:42:23.510803 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:23.510761    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" event={"ID":"d5877ad3-d063-4847-9cb3-fc33c772ee3f","Type":"ContainerStarted","Data":"53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b"}
Jul 14 17:42:23.511024 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:23.510879    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" event={"ID":"d5877ad3-d063-4847-9cb3-fc33c772ee3f","Type":"ContainerStarted","Data":"0a6b3c7715cc7339a4df6f392b3e252e53696f947824d42c2fc07eb77d4d0918"}
Jul 14 17:42:23.552815 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:23.552731    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" podStartSLOduration=1.5527089219999999 podStartE2EDuration="1.552708922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:42:23.551779111 +0000 UTC m=+2230.299379000" watchObservedRunningTime="2026-07-14 17:42:23.552708922 +0000 UTC m=+2230.300308801"
Jul 14 17:42:23.936300 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:23.936207    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:24.941903 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:24.941871    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:25.500405 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:25.500376    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:42:25.521398 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:25.521361    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:25.522261 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:25.522242    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:31.644869 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:31.644806    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:42:39.237707 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.237665    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"]
Jul 14 17:42:39.244112 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.244089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.247477 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.247454    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d44b45\"/\"openshift-service-ca.crt\""
Jul 14 17:42:39.247613 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.247478    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d44b45\"/\"default-dockercfg-4w7qd\""
Jul 14 17:42:39.248500 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.248469    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d44b45\"/\"test-model-d44b45-kserve-self-signed-certs\""
Jul 14 17:42:39.248605 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.248503    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d44b45\"/\"kube-root-ca.crt\""
Jul 14 17:42:39.252883 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.252830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"]
Jul 14 17:42:39.283964 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.283926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-home\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.283964 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.283967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l22gv\" (UniqueName: \"kubernetes.io/projected/5f73e52d-b5ab-4801-8642-6a30432dd53d-kube-api-access-l22gv\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.284202 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.283991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-model-cache\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.284202 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.284012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f73e52d-b5ab-4801-8642-6a30432dd53d-tls-certs\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.284202 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.284108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-dshm\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.284202 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.284144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-tmp-dir\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.385374 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.385333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-home\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.385590 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.385388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l22gv\" (UniqueName: \"kubernetes.io/projected/5f73e52d-b5ab-4801-8642-6a30432dd53d-kube-api-access-l22gv\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.385590 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.385422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-model-cache\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.385590 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.385453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f73e52d-b5ab-4801-8642-6a30432dd53d-tls-certs\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.385781 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.385612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-dshm\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.385781 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.385669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-tmp-dir\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.385781 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.385764    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-home\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.385987 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.385827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-model-cache\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.386043 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.385999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-tmp-dir\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.388107 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.388084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-dshm\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.388233 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.388215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f73e52d-b5ab-4801-8642-6a30432dd53d-tls-certs\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.394658 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.394635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l22gv\" (UniqueName: \"kubernetes.io/projected/5f73e52d-b5ab-4801-8642-6a30432dd53d-kube-api-access-l22gv\") pod \"test-model-d44b45-kserve-85467598c5-9fdn6\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") " pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.555466 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.555358    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:39.707414 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:39.707374    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"]
Jul 14 17:42:39.710818 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:42:39.710789    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f73e52d_b5ab_4801_8642_6a30432dd53d.slice/crio-3c96b6811af3f11454e1fbcf23b481bc2b9bc0bdaa280eb13ba2a72923e0d8c8 WatchSource:0}: Error finding container 3c96b6811af3f11454e1fbcf23b481bc2b9bc0bdaa280eb13ba2a72923e0d8c8: Status 404 returned error can't find the container with id 3c96b6811af3f11454e1fbcf23b481bc2b9bc0bdaa280eb13ba2a72923e0d8c8
Jul 14 17:42:40.522990 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.522955    2580 patch_prober.go:28] interesting pod/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 14 17:42:40.523554 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.523024    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 14 17:42:40.611580 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.611543    2580 generic.go:356] "Generic (PLEG): container finished" podID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerID="53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b" exitCode=0
Jul 14 17:42:40.611763 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.611614    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" event={"ID":"d5877ad3-d063-4847-9cb3-fc33c772ee3f","Type":"ContainerDied","Data":"53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b"}
Jul 14 17:42:40.612078 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.612056    2580 scope.go:117] "RemoveContainer" containerID="53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b"
Jul 14 17:42:40.613312 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.613274    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:42:40.613415 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.613360    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:42:40.613415 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.613288    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6" event={"ID":"5f73e52d-b5ab-4801-8642-6a30432dd53d","Type":"ContainerStarted","Data":"0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3"}
Jul 14 17:42:40.613415 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.613395    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:42:40.613415 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.613404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6" event={"ID":"5f73e52d-b5ab-4801-8642-6a30432dd53d","Type":"ContainerStarted","Data":"3c96b6811af3f11454e1fbcf23b481bc2b9bc0bdaa280eb13ba2a72923e0d8c8"}
Jul 14 17:42:40.656942 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:40.656894    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6" podStartSLOduration=1.656877582 podStartE2EDuration="1.656877582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:42:40.656032211 +0000 UTC m=+2247.403632090" watchObservedRunningTime="2026-07-14 17:42:40.656877582 +0000 UTC m=+2247.404477474"
Jul 14 17:42:41.620078 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:41.620034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" event={"ID":"d5877ad3-d063-4847-9cb3-fc33c772ee3f","Type":"ContainerStarted","Data":"46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7"}
Jul 14 17:42:41.936179 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:41.936084    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:41.936369 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:41.936344    2580 patch_prober.go:28] interesting pod/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 14 17:42:41.936425 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:41.936400    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 14 17:42:42.936696 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:42.936656    2580 patch_prober.go:28] interesting pod/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 14 17:42:42.937160 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:42.936731    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 14 17:42:43.936346 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:43.936301    2580 patch_prober.go:28] interesting pod/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 14 17:42:43.936538 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:43.936362    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 14 17:42:44.741962 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:44.741925    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:42:44.936432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:44.936398    2580 patch_prober.go:28] interesting pod/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 14 17:42:44.936616 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:44.936475    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 14 17:42:45.936322 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:45.936281    2580 patch_prober.go:28] interesting pod/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 14 17:42:45.936725 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:45.936342    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 14 17:42:46.937019 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:46.936981    2580 patch_prober.go:28] interesting pod/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 14 17:42:46.937432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:46.937036    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 14 17:42:47.966008 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:47.965975    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:47.966519 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:47.966355    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:47.967306 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:47.967285    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:42:49.556185 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:49.556142    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:49.556609 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:49.556211    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:49.568888 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:49.568830    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:49.668897 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:49.668868    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:42:57.635461 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:42:57.635425    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:43:12.342157 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:12.342112    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:43:13.164506 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:13.164464    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"]
Jul 14 17:43:24.749207 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:24.749160    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:43:24.761276 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:24.761246    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"]
Jul 14 17:43:24.761481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:24.761459    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9" podUID="b60781ff-c287-4616-a45d-a726c08552f3" containerName="maas-api" containerID="cri-o://77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a" gracePeriod=30
Jul 14 17:43:25.014080 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.014016    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:43:25.018652 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.018630    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b60781ff-c287-4616-a45d-a726c08552f3-maas-api-tls\") pod \"b60781ff-c287-4616-a45d-a726c08552f3\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") "
Jul 14 17:43:25.018750 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.018668    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzbxp\" (UniqueName: \"kubernetes.io/projected/b60781ff-c287-4616-a45d-a726c08552f3-kube-api-access-rzbxp\") pod \"b60781ff-c287-4616-a45d-a726c08552f3\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") "
Jul 14 17:43:25.018750 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.018734    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60781ff-c287-4616-a45d-a726c08552f3-openshift-service-ca\") pod \"b60781ff-c287-4616-a45d-a726c08552f3\" (UID: \"b60781ff-c287-4616-a45d-a726c08552f3\") "
Jul 14 17:43:25.019192 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.019167    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b60781ff-c287-4616-a45d-a726c08552f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b60781ff-c287-4616-a45d-a726c08552f3" (UID: "b60781ff-c287-4616-a45d-a726c08552f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:43:25.020879 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.020834    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b60781ff-c287-4616-a45d-a726c08552f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b60781ff-c287-4616-a45d-a726c08552f3" (UID: "b60781ff-c287-4616-a45d-a726c08552f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:43:25.020879 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.020866    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b60781ff-c287-4616-a45d-a726c08552f3-kube-api-access-rzbxp" (OuterVolumeSpecName: "kube-api-access-rzbxp") pod "b60781ff-c287-4616-a45d-a726c08552f3" (UID: "b60781ff-c287-4616-a45d-a726c08552f3"). InnerVolumeSpecName "kube-api-access-rzbxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:43:25.119401 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.119368    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60781ff-c287-4616-a45d-a726c08552f3-openshift-service-ca\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:43:25.119401 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.119398    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b60781ff-c287-4616-a45d-a726c08552f3-maas-api-tls\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:43:25.119401 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.119407    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzbxp\" (UniqueName: \"kubernetes.io/projected/b60781ff-c287-4616-a45d-a726c08552f3-kube-api-access-rzbxp\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:43:25.838750 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.838709    2580 generic.go:356] "Generic (PLEG): container finished" podID="b60781ff-c287-4616-a45d-a726c08552f3" containerID="77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a" exitCode=0
Jul 14 17:43:25.838750 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.838752    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"
Jul 14 17:43:25.839227 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.838746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9" event={"ID":"b60781ff-c287-4616-a45d-a726c08552f3","Type":"ContainerDied","Data":"77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a"}
Jul 14 17:43:25.839227 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.838889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9" event={"ID":"b60781ff-c287-4616-a45d-a726c08552f3","Type":"ContainerDied","Data":"4f4d5ec87f0872d446fe6cb6b4c7737bf71456f22c6180812417b280b2835806"}
Jul 14 17:43:25.839227 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.838920    2580 scope.go:117] "RemoveContainer" containerID="77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a"
Jul 14 17:43:25.848609 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.848588    2580 scope.go:117] "RemoveContainer" containerID="77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a"
Jul 14 17:43:25.848912 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:43:25.848893    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a\": container with ID starting with 77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a not found: ID does not exist" containerID="77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a"
Jul 14 17:43:25.848987 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.848928    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a"} err="failed to get container status \"77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a\": rpc error: code = NotFound desc = could not find container \"77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a\": container with ID starting with 77deba1f0f7f2bf6a43d2b77e1f4bc4af09fb35d7d05d37f659dc98accf8e88a not found: ID does not exist"
Jul 14 17:43:25.863142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.863118    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"]
Jul 14 17:43:25.867744 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.867720    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d44b45-58cf46d996-kmbs9"]
Jul 14 17:43:25.881970 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:25.881946    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b60781ff-c287-4616-a45d-a726c08552f3" path="/var/lib/kubelet/pods/b60781ff-c287-4616-a45d-a726c08552f3/volumes"
Jul 14 17:43:28.185901 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.185814    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6" podUID="5f73e52d-b5ab-4801-8642-6a30432dd53d" containerName="main" containerID="cri-o://0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3" gracePeriod=45
Jul 14 17:43:28.446123 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.446101    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:43:28.543253 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543216    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l22gv\" (UniqueName: \"kubernetes.io/projected/5f73e52d-b5ab-4801-8642-6a30432dd53d-kube-api-access-l22gv\") pod \"5f73e52d-b5ab-4801-8642-6a30432dd53d\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") "
Jul 14 17:43:28.543253 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543262    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-model-cache\") pod \"5f73e52d-b5ab-4801-8642-6a30432dd53d\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") "
Jul 14 17:43:28.543517 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543280    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-tmp-dir\") pod \"5f73e52d-b5ab-4801-8642-6a30432dd53d\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") "
Jul 14 17:43:28.543517 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543320    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-dshm\") pod \"5f73e52d-b5ab-4801-8642-6a30432dd53d\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") "
Jul 14 17:43:28.543517 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543357    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-home\") pod \"5f73e52d-b5ab-4801-8642-6a30432dd53d\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") "
Jul 14 17:43:28.543517 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543387    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f73e52d-b5ab-4801-8642-6a30432dd53d-tls-certs\") pod \"5f73e52d-b5ab-4801-8642-6a30432dd53d\" (UID: \"5f73e52d-b5ab-4801-8642-6a30432dd53d\") "
Jul 14 17:43:28.543724 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543584    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5f73e52d-b5ab-4801-8642-6a30432dd53d" (UID: "5f73e52d-b5ab-4801-8642-6a30432dd53d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:43:28.543724 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543618    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-model-cache" (OuterVolumeSpecName: "model-cache") pod "5f73e52d-b5ab-4801-8642-6a30432dd53d" (UID: "5f73e52d-b5ab-4801-8642-6a30432dd53d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:43:28.543826 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-home" (OuterVolumeSpecName: "home") pod "5f73e52d-b5ab-4801-8642-6a30432dd53d" (UID: "5f73e52d-b5ab-4801-8642-6a30432dd53d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:43:28.543826 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543773    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-model-cache\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:43:28.543826 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.543791    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-tmp-dir\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:43:28.545795 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.545766    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-dshm" (OuterVolumeSpecName: "dshm") pod "5f73e52d-b5ab-4801-8642-6a30432dd53d" (UID: "5f73e52d-b5ab-4801-8642-6a30432dd53d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:43:28.545949 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.545802    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f73e52d-b5ab-4801-8642-6a30432dd53d-kube-api-access-l22gv" (OuterVolumeSpecName: "kube-api-access-l22gv") pod "5f73e52d-b5ab-4801-8642-6a30432dd53d" (UID: "5f73e52d-b5ab-4801-8642-6a30432dd53d"). InnerVolumeSpecName "kube-api-access-l22gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:43:28.545949 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.545864    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f73e52d-b5ab-4801-8642-6a30432dd53d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5f73e52d-b5ab-4801-8642-6a30432dd53d" (UID: "5f73e52d-b5ab-4801-8642-6a30432dd53d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:43:28.644583 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.644550    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l22gv\" (UniqueName: \"kubernetes.io/projected/5f73e52d-b5ab-4801-8642-6a30432dd53d-kube-api-access-l22gv\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:43:28.644583 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.644580    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-dshm\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:43:28.644583 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.644592    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f73e52d-b5ab-4801-8642-6a30432dd53d-home\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:43:28.644817 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.644600    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f73e52d-b5ab-4801-8642-6a30432dd53d-tls-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:43:28.857288 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.857189    2580 generic.go:356] "Generic (PLEG): container finished" podID="5f73e52d-b5ab-4801-8642-6a30432dd53d" containerID="0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3" exitCode=0
Jul 14 17:43:28.857288 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.857230    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"
Jul 14 17:43:28.857288 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.857274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6" event={"ID":"5f73e52d-b5ab-4801-8642-6a30432dd53d","Type":"ContainerDied","Data":"0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3"}
Jul 14 17:43:28.857573 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.857320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6" event={"ID":"5f73e52d-b5ab-4801-8642-6a30432dd53d","Type":"ContainerDied","Data":"3c96b6811af3f11454e1fbcf23b481bc2b9bc0bdaa280eb13ba2a72923e0d8c8"}
Jul 14 17:43:28.857573 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.857338    2580 scope.go:117] "RemoveContainer" containerID="0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3"
Jul 14 17:43:28.866810 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.866792    2580 scope.go:117] "RemoveContainer" containerID="0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3"
Jul 14 17:43:28.867105 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:43:28.867088    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3\": container with ID starting with 0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3 not found: ID does not exist" containerID="0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3"
Jul 14 17:43:28.867166 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.867113    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3"} err="failed to get container status \"0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3\": rpc error: code = NotFound desc = could not find container \"0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3\": container with ID starting with 0abe908fc3f5fe3a33003110a40be45a979fa9f130f2f47f07e13f24276de6f3 not found: ID does not exist"
Jul 14 17:43:28.880638 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.880612    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"]
Jul 14 17:43:28.884732 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:28.884705    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d44b45/test-model-d44b45-kserve-85467598c5-9fdn6"]
Jul 14 17:43:29.882753 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:29.882722    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f73e52d-b5ab-4801-8642-6a30432dd53d" path="/var/lib/kubelet/pods/5f73e52d-b5ab-4801-8642-6a30432dd53d/volumes"
Jul 14 17:43:34.758140 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:34.758101    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:43:54.087237 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:43:54.087199    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:44:06.281567 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:06.281484    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"]
Jul 14 17:44:06.282092 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:06.281741    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy" containerID="cri-o://46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7" gracePeriod=30
Jul 14 17:44:10.208432 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.208396    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"]
Jul 14 17:44:10.208940 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.208866    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f73e52d-b5ab-4801-8642-6a30432dd53d" containerName="main"
Jul 14 17:44:10.208940 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.208883    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f73e52d-b5ab-4801-8642-6a30432dd53d" containerName="main"
Jul 14 17:44:10.208940 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.208908    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b60781ff-c287-4616-a45d-a726c08552f3" containerName="maas-api"
Jul 14 17:44:10.208940 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.208914    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b60781ff-c287-4616-a45d-a726c08552f3" containerName="maas-api"
Jul 14 17:44:10.209157 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.208987    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f73e52d-b5ab-4801-8642-6a30432dd53d" containerName="main"
Jul 14 17:44:10.209157 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.208994    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b60781ff-c287-4616-a45d-a726c08552f3" containerName="maas-api"
Jul 14 17:44:10.212279 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.212258    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:10.215040 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.215019    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-083b58-serving-cert\""
Jul 14 17:44:10.233057 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.233028    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"]
Jul 14 17:44:10.319457 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.319425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfrph\" (UniqueName: \"kubernetes.io/projected/a47ce8f4-bc24-44b3-b65a-aafd580a997a-kube-api-access-gfrph\") pod \"maas-api-e2e-api-a-083b58-5d9dcff4c6-87572\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:10.319630 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.319538    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47ce8f4-bc24-44b3-b65a-aafd580a997a-maas-api-tls\") pod \"maas-api-e2e-api-a-083b58-5d9dcff4c6-87572\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:10.319630 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.319598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47ce8f4-bc24-44b3-b65a-aafd580a997a-openshift-service-ca\") pod \"maas-api-e2e-api-a-083b58-5d9dcff4c6-87572\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:10.420882 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.420831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47ce8f4-bc24-44b3-b65a-aafd580a997a-maas-api-tls\") pod \"maas-api-e2e-api-a-083b58-5d9dcff4c6-87572\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:10.421096 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.420900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47ce8f4-bc24-44b3-b65a-aafd580a997a-openshift-service-ca\") pod \"maas-api-e2e-api-a-083b58-5d9dcff4c6-87572\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:10.421096 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.420938    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfrph\" (UniqueName: \"kubernetes.io/projected/a47ce8f4-bc24-44b3-b65a-aafd580a997a-kube-api-access-gfrph\") pod \"maas-api-e2e-api-a-083b58-5d9dcff4c6-87572\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:10.421096 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:44:10.420989    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-serving-cert: secret "maas-api-e2e-api-a-083b58-serving-cert" not found
Jul 14 17:44:10.421096 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:44:10.421072    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a47ce8f4-bc24-44b3-b65a-aafd580a997a-maas-api-tls podName:a47ce8f4-bc24-44b3-b65a-aafd580a997a nodeName:}" failed. No retries permitted until 2026-07-14 17:44:10.921055516 +0000 UTC m=+2337.668655378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a47ce8f4-bc24-44b3-b65a-aafd580a997a-maas-api-tls") pod "maas-api-e2e-api-a-083b58-5d9dcff4c6-87572" (UID: "a47ce8f4-bc24-44b3-b65a-aafd580a997a") : secret "maas-api-e2e-api-a-083b58-serving-cert" not found
Jul 14 17:44:10.421648 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.421620    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47ce8f4-bc24-44b3-b65a-aafd580a997a-openshift-service-ca\") pod \"maas-api-e2e-api-a-083b58-5d9dcff4c6-87572\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:10.439028 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.438998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfrph\" (UniqueName: \"kubernetes.io/projected/a47ce8f4-bc24-44b3-b65a-aafd580a997a-kube-api-access-gfrph\") pod \"maas-api-e2e-api-a-083b58-5d9dcff4c6-87572\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:10.925754 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.925708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47ce8f4-bc24-44b3-b65a-aafd580a997a-maas-api-tls\") pod \"maas-api-e2e-api-a-083b58-5d9dcff4c6-87572\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:10.928355 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:10.928322    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47ce8f4-bc24-44b3-b65a-aafd580a997a-maas-api-tls\") pod \"maas-api-e2e-api-a-083b58-5d9dcff4c6-87572\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:11.122404 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.122365    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:11.270420 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.270381    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"]
Jul 14 17:44:11.273822 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:44:11.273795    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda47ce8f4_bc24_44b3_b65a_aafd580a997a.slice/crio-db528b90c8e8deece91b47c69a179010055955c5193e92f1264aa41b8a20c364 WatchSource:0}: Error finding container db528b90c8e8deece91b47c69a179010055955c5193e92f1264aa41b8a20c364: Status 404 returned error can't find the container with id db528b90c8e8deece91b47c69a179010055955c5193e92f1264aa41b8a20c364
Jul 14 17:44:11.275208 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.275190    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 17:44:11.555151 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.555124    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:44:11.734249 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734159    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v67hj\" (UniqueName: \"kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-kube-api-access-v67hj\") pod \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") "
Jul 14 17:44:11.734249 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734211    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-certs\") pod \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") "
Jul 14 17:44:11.734475 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734255    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-data\") pod \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") "
Jul 14 17:44:11.734475 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734291    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-token\") pod \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") "
Jul 14 17:44:11.734475 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734309    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-podinfo\") pod \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") "
Jul 14 17:44:11.734475 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734361    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-credential-socket\") pod \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") "
Jul 14 17:44:11.734475 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734387    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istiod-ca-cert\") pod \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") "
Jul 14 17:44:11.734475 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734420    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-socket\") pod \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") "
Jul 14 17:44:11.734475 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734450    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-envoy\") pod \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\" (UID: \"d5877ad3-d063-4847-9cb3-fc33c772ee3f\") "
Jul 14 17:44:11.734866 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734582    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d5877ad3-d063-4847-9cb3-fc33c772ee3f" (UID: "d5877ad3-d063-4847-9cb3-fc33c772ee3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:44:11.734866 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734671    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d5877ad3-d063-4847-9cb3-fc33c772ee3f" (UID: "d5877ad3-d063-4847-9cb3-fc33c772ee3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:44:11.734866 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734801    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:11.734866 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734808    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d5877ad3-d063-4847-9cb3-fc33c772ee3f" (UID: "d5877ad3-d063-4847-9cb3-fc33c772ee3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:44:11.734866 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734821    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-credential-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:11.735148 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.734938    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d5877ad3-d063-4847-9cb3-fc33c772ee3f" (UID: "d5877ad3-d063-4847-9cb3-fc33c772ee3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:44:11.735587 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.735557    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d5877ad3-d063-4847-9cb3-fc33c772ee3f" (UID: "d5877ad3-d063-4847-9cb3-fc33c772ee3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:44:11.736672 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.736642    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d5877ad3-d063-4847-9cb3-fc33c772ee3f" (UID: "d5877ad3-d063-4847-9cb3-fc33c772ee3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 17:44:11.736672 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.736664    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d5877ad3-d063-4847-9cb3-fc33c772ee3f" (UID: "d5877ad3-d063-4847-9cb3-fc33c772ee3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:44:11.736837 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.736729    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d5877ad3-d063-4847-9cb3-fc33c772ee3f" (UID: "d5877ad3-d063-4847-9cb3-fc33c772ee3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:44:11.736967 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.736949    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-kube-api-access-v67hj" (OuterVolumeSpecName: "kube-api-access-v67hj") pod "d5877ad3-d063-4847-9cb3-fc33c772ee3f" (UID: "d5877ad3-d063-4847-9cb3-fc33c772ee3f"). InnerVolumeSpecName "kube-api-access-v67hj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:44:11.835823 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.835773    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-token\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:11.835823 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.835822    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-podinfo\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:11.835823 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.835833    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istiod-ca-cert\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:11.835823 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.835863    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-workload-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:11.835823 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.835873    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-envoy\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:11.836154 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.835881    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v67hj\" (UniqueName: \"kubernetes.io/projected/d5877ad3-d063-4847-9cb3-fc33c772ee3f-kube-api-access-v67hj\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:11.836154 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:11.835890    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5877ad3-d063-4847-9cb3-fc33c772ee3f-istio-data\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:12.073002 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.072891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572" event={"ID":"a47ce8f4-bc24-44b3-b65a-aafd580a997a","Type":"ContainerStarted","Data":"2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52"}
Jul 14 17:44:12.073002 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.072959    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:12.073002 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.072982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572" event={"ID":"a47ce8f4-bc24-44b3-b65a-aafd580a997a","Type":"ContainerStarted","Data":"db528b90c8e8deece91b47c69a179010055955c5193e92f1264aa41b8a20c364"}
Jul 14 17:44:12.074528 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.074500    2580 generic.go:356] "Generic (PLEG): container finished" podID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerID="46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7" exitCode=0
Jul 14 17:44:12.074641 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.074542    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"
Jul 14 17:44:12.074641 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.074573    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" event={"ID":"d5877ad3-d063-4847-9cb3-fc33c772ee3f","Type":"ContainerDied","Data":"46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7"}
Jul 14 17:44:12.074641 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.074614    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89" event={"ID":"d5877ad3-d063-4847-9cb3-fc33c772ee3f","Type":"ContainerDied","Data":"0a6b3c7715cc7339a4df6f392b3e252e53696f947824d42c2fc07eb77d4d0918"}
Jul 14 17:44:12.074747 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.074642    2580 scope.go:117] "RemoveContainer" containerID="46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7"
Jul 14 17:44:12.083628 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.083597    2580 scope.go:117] "RemoveContainer" containerID="53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b"
Jul 14 17:44:12.091959 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.091939    2580 scope.go:117] "RemoveContainer" containerID="46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7"
Jul 14 17:44:12.092260 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:44:12.092235    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7\": container with ID starting with 46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7 not found: ID does not exist" containerID="46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7"
Jul 14 17:44:12.092355 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.092270    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7"} err="failed to get container status \"46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7\": rpc error: code = NotFound desc = could not find container \"46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7\": container with ID starting with 46b8c7acc68248848f20375f8c33bc45285611f46db93a2699c40af0fc6b29c7 not found: ID does not exist"
Jul 14 17:44:12.092355 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.092296    2580 scope.go:117] "RemoveContainer" containerID="53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b"
Jul 14 17:44:12.092592 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:44:12.092568    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b\": container with ID starting with 53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b not found: ID does not exist" containerID="53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b"
Jul 14 17:44:12.092638 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.092603    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b"} err="failed to get container status \"53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b\": rpc error: code = NotFound desc = could not find container \"53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b\": container with ID starting with 53904459615a6b0ac273cd4298f9b44d4633f23f87fd4dfc85c9ee2872c2be9b not found: ID does not exist"
Jul 14 17:44:12.097194 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.097151    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572" podStartSLOduration=1.6223144440000001 podStartE2EDuration="2.097130034s" totalImagesPullingTime="474.81559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:44:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:44:12.095738921 +0000 UTC m=+2338.843338829" watchObservedRunningTime="2026-07-14 17:44:12.097130034 +0000 UTC m=+2338.844729910"
Jul 14 17:44:12.112584 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.112544    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"]
Jul 14 17:44:12.114630 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:12.114601    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1b06a1-openshift-default-5868ff5fbf-j7t89"]
Jul 14 17:44:13.882086 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:13.882049    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" path="/var/lib/kubelet/pods/d5877ad3-d063-4847-9cb3-fc33c772ee3f/volumes"
Jul 14 17:44:14.445050 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:14.445012    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:44:15.302916 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.302882    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"]
Jul 14 17:44:15.303335 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.303311    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy"
Jul 14 17:44:15.303335 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.303328    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy"
Jul 14 17:44:15.303455 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.303340    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy"
Jul 14 17:44:15.303455 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.303349    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy"
Jul 14 17:44:15.303563 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.303478    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy"
Jul 14 17:44:15.308467 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.308446    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.311173 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.311154    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-6f8249-openshift-default-dockercfg-46hnw\""
Jul 14 17:44:15.322396 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.322374    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"]
Jul 14 17:44:15.360741 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.360701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-credential-socket\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.360951 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.360766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-socket\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.360951 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.360788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-envoy\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.360951 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.360869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-certs\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.360951 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.360910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29008bf3-76c0-47da-b097-1281743baf2b-istiod-ca-cert\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.360951 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.360947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-istio-token\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.361172 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.360995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29008bf3-76c0-47da-b097-1281743baf2b-istio-podinfo\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.361172 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.361031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcgz7\" (UniqueName: \"kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-kube-api-access-wcgz7\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.361172 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.361060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-data\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462210 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-socket\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462210 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-envoy\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462448 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-certs\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462448 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29008bf3-76c0-47da-b097-1281743baf2b-istiod-ca-cert\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462448 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-istio-token\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462601 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29008bf3-76c0-47da-b097-1281743baf2b-istio-podinfo\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462664 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462631    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-certs\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462713 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcgz7\" (UniqueName: \"kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-kube-api-access-wcgz7\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462713 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462689    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-socket\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462881 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-data\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.462881 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.462785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-credential-socket\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.463032 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.463011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-data\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.463118 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.463098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-credential-socket\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.463245 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.463228    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29008bf3-76c0-47da-b097-1281743baf2b-istiod-ca-cert\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.465413 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.465390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-envoy\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.465624 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.465606    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29008bf3-76c0-47da-b097-1281743baf2b-istio-podinfo\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.475526 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.475497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-istio-token\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.475668 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.475608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcgz7\" (UniqueName: \"kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-kube-api-access-wcgz7\") pod \"e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") " pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.618545 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.618451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:15.767079 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.767040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"]
Jul 14 17:44:15.770566 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:44:15.770538    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29008bf3_76c0_47da_b097_1281743baf2b.slice/crio-102dcce6a04687d182b6b3e7327a468ac35f24f7cef6c7ed60a8fe8611e712d6 WatchSource:0}: Error finding container 102dcce6a04687d182b6b3e7327a468ac35f24f7cef6c7ed60a8fe8611e712d6: Status 404 returned error can't find the container with id 102dcce6a04687d182b6b3e7327a468ac35f24f7cef6c7ed60a8fe8611e712d6
Jul 14 17:44:15.773001 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.772961    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:44:15.773130 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.773054    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:44:15.773130 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:15.773101    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:44:16.095974 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:16.095941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" event={"ID":"29008bf3-76c0-47da-b097-1281743baf2b","Type":"ContainerStarted","Data":"f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc"}
Jul 14 17:44:16.096146 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:16.095997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" event={"ID":"29008bf3-76c0-47da-b097-1281743baf2b","Type":"ContainerStarted","Data":"102dcce6a04687d182b6b3e7327a468ac35f24f7cef6c7ed60a8fe8611e712d6"}
Jul 14 17:44:16.128095 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:16.128042    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" podStartSLOduration=1.128020316 podStartE2EDuration="1.128020316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:44:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:44:16.127501953 +0000 UTC m=+2342.875101850" watchObservedRunningTime="2026-07-14 17:44:16.128020316 +0000 UTC m=+2342.875620170"
Jul 14 17:44:16.619243 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:16.619199    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:16.624219 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:16.624189    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:17.101235 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:17.101206    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:17.102395 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:17.102367    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:18.086117 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:18.086087    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:24.937871 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:24.937815    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:44:33.142077 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:33.141975    2580 patch_prober.go:28] interesting pod/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 14 17:44:33.142077 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:33.142063    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 14 17:44:34.188578 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:34.188539    2580 generic.go:356] "Generic (PLEG): container finished" podID="29008bf3-76c0-47da-b097-1281743baf2b" containerID="f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc" exitCode=0
Jul 14 17:44:34.189059 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:34.188605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" event={"ID":"29008bf3-76c0-47da-b097-1281743baf2b","Type":"ContainerDied","Data":"f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc"}
Jul 14 17:44:34.189059 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:34.189008    2580 scope.go:117] "RemoveContainer" containerID="f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc"
Jul 14 17:44:34.190133 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:34.190101    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:44:34.190248 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:34.190173    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:44:34.190248 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:34.190204    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:44:35.198372 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:35.198340    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" event={"ID":"29008bf3-76c0-47da-b097-1281743baf2b","Type":"ContainerStarted","Data":"ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd"}
Jul 14 17:44:35.618919 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:35.618772    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:35.619078 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:35.619004    2580 patch_prober.go:28] interesting pod/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 14 17:44:35.619078 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:35.619064    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 14 17:44:36.619346 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:36.619310    2580 patch_prober.go:28] interesting pod/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 14 17:44:36.619776 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:36.619368    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 14 17:44:37.619339 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:37.619299    2580 patch_prober.go:28] interesting pod/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 14 17:44:37.619599 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:37.619352    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 14 17:44:38.619082 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:38.619034    2580 patch_prober.go:28] interesting pod/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 14 17:44:38.619266 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:38.619103    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 14 17:44:39.623005 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:39.622976    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:39.623378 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:39.623247    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:39.623914 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:39.623896    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:44:42.715249 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:42.715211    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"]
Jul 14 17:44:42.715669 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:42.715443    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572" podUID="a47ce8f4-bc24-44b3-b65a-aafd580a997a" containerName="maas-api" containerID="cri-o://2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52" gracePeriod=30
Jul 14 17:44:42.974677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:42.974608    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:43.029568 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.029539    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfrph\" (UniqueName: \"kubernetes.io/projected/a47ce8f4-bc24-44b3-b65a-aafd580a997a-kube-api-access-gfrph\") pod \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") "
Jul 14 17:44:43.029762 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.029603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47ce8f4-bc24-44b3-b65a-aafd580a997a-openshift-service-ca\") pod \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") "
Jul 14 17:44:43.029762 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.029682    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47ce8f4-bc24-44b3-b65a-aafd580a997a-maas-api-tls\") pod \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\" (UID: \"a47ce8f4-bc24-44b3-b65a-aafd580a997a\") "
Jul 14 17:44:43.030106 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.030069    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a47ce8f4-bc24-44b3-b65a-aafd580a997a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a47ce8f4-bc24-44b3-b65a-aafd580a997a" (UID: "a47ce8f4-bc24-44b3-b65a-aafd580a997a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:44:43.032031 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.032001    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a47ce8f4-bc24-44b3-b65a-aafd580a997a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a47ce8f4-bc24-44b3-b65a-aafd580a997a" (UID: "a47ce8f4-bc24-44b3-b65a-aafd580a997a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:44:43.032369 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.032345    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a47ce8f4-bc24-44b3-b65a-aafd580a997a-kube-api-access-gfrph" (OuterVolumeSpecName: "kube-api-access-gfrph") pod "a47ce8f4-bc24-44b3-b65a-aafd580a997a" (UID: "a47ce8f4-bc24-44b3-b65a-aafd580a997a"). InnerVolumeSpecName "kube-api-access-gfrph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:44:43.130813 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.130766    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47ce8f4-bc24-44b3-b65a-aafd580a997a-maas-api-tls\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:43.130813 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.130798    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfrph\" (UniqueName: \"kubernetes.io/projected/a47ce8f4-bc24-44b3-b65a-aafd580a997a-kube-api-access-gfrph\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:43.130813 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.130810    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47ce8f4-bc24-44b3-b65a-aafd580a997a-openshift-service-ca\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:44:43.236368 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.236276    2580 generic.go:356] "Generic (PLEG): container finished" podID="a47ce8f4-bc24-44b3-b65a-aafd580a997a" containerID="2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52" exitCode=0
Jul 14 17:44:43.236368 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.236322    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"
Jul 14 17:44:43.236569 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.236366    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572" event={"ID":"a47ce8f4-bc24-44b3-b65a-aafd580a997a","Type":"ContainerDied","Data":"2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52"}
Jul 14 17:44:43.236569 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.236415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572" event={"ID":"a47ce8f4-bc24-44b3-b65a-aafd580a997a","Type":"ContainerDied","Data":"db528b90c8e8deece91b47c69a179010055955c5193e92f1264aa41b8a20c364"}
Jul 14 17:44:43.236569 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.236438    2580 scope.go:117] "RemoveContainer" containerID="2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52"
Jul 14 17:44:43.249976 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.247438    2580 scope.go:117] "RemoveContainer" containerID="2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52"
Jul 14 17:44:43.249976 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:44:43.248333    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52\": container with ID starting with 2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52 not found: ID does not exist" containerID="2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52"
Jul 14 17:44:43.249976 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.248367    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52"} err="failed to get container status \"2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52\": rpc error: code = NotFound desc = could not find container \"2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52\": container with ID starting with 2a48cec9117f1d8a091812bb9f7dffb1570b366e7f9b17f93c62cc964fcc7c52 not found: ID does not exist"
Jul 14 17:44:43.267358 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.267328    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"]
Jul 14 17:44:43.273262 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.273230    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-083b58-5d9dcff4c6-87572"]
Jul 14 17:44:43.882250 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:43.882217    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a47ce8f4-bc24-44b3-b65a-aafd580a997a" path="/var/lib/kubelet/pods/a47ce8f4-bc24-44b3-b65a-aafd580a997a/volumes"
Jul 14 17:44:46.651423 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:44:46.651383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:45:10.856256 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:10.856216    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:45:13.875082 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:13.875050    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:45:13.883294 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:13.883271    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:45:22.688027 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:22.687985    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"]
Jul 14 17:45:22.688548 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:22.688256    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy" containerID="cri-o://ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd" gracePeriod=30
Jul 14 17:45:24.625942 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:24.625901    2580 patch_prober.go:28] interesting pod/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 14 17:45:24.626313 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:24.625967    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 17:45:26.473862 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.473817    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"]
Jul 14 17:45:26.474255 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.474205    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a47ce8f4-bc24-44b3-b65a-aafd580a997a" containerName="maas-api"
Jul 14 17:45:26.474255 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.474216    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a47ce8f4-bc24-44b3-b65a-aafd580a997a" containerName="maas-api"
Jul 14 17:45:26.474363 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.474281    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5877ad3-d063-4847-9cb3-fc33c772ee3f" containerName="istio-proxy"
Jul 14 17:45:26.474363 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.474289    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a47ce8f4-bc24-44b3-b65a-aafd580a997a" containerName="maas-api"
Jul 14 17:45:26.476443 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.476423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:26.479167 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.479143    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-246d80-serving-cert\""
Jul 14 17:45:26.496804 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.496765    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"]
Jul 14 17:45:26.534700 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.534668    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a9a556-24a9-46fd-afb9-38b324380a16-maas-api-tls\") pod \"maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:26.534924 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.534723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a9a556-24a9-46fd-afb9-38b324380a16-openshift-service-ca\") pod \"maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:26.534924 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.534746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r7f4\" (UniqueName: \"kubernetes.io/projected/49a9a556-24a9-46fd-afb9-38b324380a16-kube-api-access-4r7f4\") pod \"maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:26.635493 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.635446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a9a556-24a9-46fd-afb9-38b324380a16-maas-api-tls\") pod \"maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:26.635674 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.635513    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a9a556-24a9-46fd-afb9-38b324380a16-openshift-service-ca\") pod \"maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:26.635674 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.635546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r7f4\" (UniqueName: \"kubernetes.io/projected/49a9a556-24a9-46fd-afb9-38b324380a16-kube-api-access-4r7f4\") pod \"maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:26.635674 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:45:26.635618    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-serving-cert: secret "maas-api-e2e-shared-a-246d80-serving-cert" not found
Jul 14 17:45:26.635776 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:45:26.635714    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49a9a556-24a9-46fd-afb9-38b324380a16-maas-api-tls podName:49a9a556-24a9-46fd-afb9-38b324380a16 nodeName:}" failed. No retries permitted until 2026-07-14 17:45:27.135690637 +0000 UTC m=+2413.883290507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/49a9a556-24a9-46fd-afb9-38b324380a16-maas-api-tls") pod "maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q" (UID: "49a9a556-24a9-46fd-afb9-38b324380a16") : secret "maas-api-e2e-shared-a-246d80-serving-cert" not found
Jul 14 17:45:26.636238 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.636215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a9a556-24a9-46fd-afb9-38b324380a16-openshift-service-ca\") pod \"maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:26.654193 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:26.654168    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r7f4\" (UniqueName: \"kubernetes.io/projected/49a9a556-24a9-46fd-afb9-38b324380a16-kube-api-access-4r7f4\") pod \"maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:27.140481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:27.140441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a9a556-24a9-46fd-afb9-38b324380a16-maas-api-tls\") pod \"maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:27.143074 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:27.143042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a9a556-24a9-46fd-afb9-38b324380a16-maas-api-tls\") pod \"maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:27.387669 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:27.387634    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:27.533365 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:27.533330    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"]
Jul 14 17:45:27.536580 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:45:27.536550    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49a9a556_24a9_46fd_afb9_38b324380a16.slice/crio-040e4015c4708d3bb8f8d59e1424cab5ef0a4ca4cfd93d6498929ee2436e1cb3 WatchSource:0}: Error finding container 040e4015c4708d3bb8f8d59e1424cab5ef0a4ca4cfd93d6498929ee2436e1cb3: Status 404 returned error can't find the container with id 040e4015c4708d3bb8f8d59e1424cab5ef0a4ca4cfd93d6498929ee2436e1cb3
Jul 14 17:45:27.959162 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:27.959101    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:45:28.049677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.049644    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-credential-socket\") pod \"29008bf3-76c0-47da-b097-1281743baf2b\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") "
Jul 14 17:45:28.049677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.049683    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-certs\") pod \"29008bf3-76c0-47da-b097-1281743baf2b\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") "
Jul 14 17:45:28.049989 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.049714    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcgz7\" (UniqueName: \"kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-kube-api-access-wcgz7\") pod \"29008bf3-76c0-47da-b097-1281743baf2b\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") "
Jul 14 17:45:28.049989 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.049746    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-data\") pod \"29008bf3-76c0-47da-b097-1281743baf2b\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") "
Jul 14 17:45:28.049989 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.049776    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-socket\") pod \"29008bf3-76c0-47da-b097-1281743baf2b\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") "
Jul 14 17:45:28.049989 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.049821    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-envoy\") pod \"29008bf3-76c0-47da-b097-1281743baf2b\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") "
Jul 14 17:45:28.050211 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050019    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29008bf3-76c0-47da-b097-1281743baf2b-istiod-ca-cert\") pod \"29008bf3-76c0-47da-b097-1281743baf2b\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") "
Jul 14 17:45:28.050211 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050050    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29008bf3-76c0-47da-b097-1281743baf2b-istio-podinfo\") pod \"29008bf3-76c0-47da-b097-1281743baf2b\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") "
Jul 14 17:45:28.050211 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050037    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29008bf3-76c0-47da-b097-1281743baf2b" (UID: "29008bf3-76c0-47da-b097-1281743baf2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:45:28.050211 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050087    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29008bf3-76c0-47da-b097-1281743baf2b" (UID: "29008bf3-76c0-47da-b097-1281743baf2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:45:28.050211 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050107    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-istio-token\") pod \"29008bf3-76c0-47da-b097-1281743baf2b\" (UID: \"29008bf3-76c0-47da-b097-1281743baf2b\") "
Jul 14 17:45:28.050211 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050162    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29008bf3-76c0-47da-b097-1281743baf2b" (UID: "29008bf3-76c0-47da-b097-1281743baf2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:45:28.050514 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050388    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29008bf3-76c0-47da-b097-1281743baf2b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29008bf3-76c0-47da-b097-1281743baf2b" (UID: "29008bf3-76c0-47da-b097-1281743baf2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:45:28.050514 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050484    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29008bf3-76c0-47da-b097-1281743baf2b-istiod-ca-cert\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:45:28.050514 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050501    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-credential-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:45:28.050655 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050515    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:45:28.050655 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050529    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-workload-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:45:28.050945 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.050914    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-data" (OuterVolumeSpecName: "istio-data") pod "29008bf3-76c0-47da-b097-1281743baf2b" (UID: "29008bf3-76c0-47da-b097-1281743baf2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:45:28.052329 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.052300    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-kube-api-access-wcgz7" (OuterVolumeSpecName: "kube-api-access-wcgz7") pod "29008bf3-76c0-47da-b097-1281743baf2b" (UID: "29008bf3-76c0-47da-b097-1281743baf2b"). InnerVolumeSpecName "kube-api-access-wcgz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:45:28.052450 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.052328    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29008bf3-76c0-47da-b097-1281743baf2b" (UID: "29008bf3-76c0-47da-b097-1281743baf2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:45:28.052450 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.052333    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29008bf3-76c0-47da-b097-1281743baf2b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29008bf3-76c0-47da-b097-1281743baf2b" (UID: "29008bf3-76c0-47da-b097-1281743baf2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 17:45:28.052536 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.052515    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-istio-token" (OuterVolumeSpecName: "istio-token") pod "29008bf3-76c0-47da-b097-1281743baf2b" (UID: "29008bf3-76c0-47da-b097-1281743baf2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:45:28.157664 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.157591    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-istio-token\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:45:28.157664 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.157633    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcgz7\" (UniqueName: \"kubernetes.io/projected/29008bf3-76c0-47da-b097-1281743baf2b-kube-api-access-wcgz7\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:45:28.157664 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.157647    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-data\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:45:28.157987 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.157868    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29008bf3-76c0-47da-b097-1281743baf2b-istio-envoy\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:45:28.158042 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.157992    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29008bf3-76c0-47da-b097-1281743baf2b-istio-podinfo\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:45:28.457505 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.457467    2580 generic.go:356] "Generic (PLEG): container finished" podID="29008bf3-76c0-47da-b097-1281743baf2b" containerID="ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd" exitCode=0
Jul 14 17:45:28.457661 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.457533    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"
Jul 14 17:45:28.457661 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.457556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" event={"ID":"29008bf3-76c0-47da-b097-1281743baf2b","Type":"ContainerDied","Data":"ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd"}
Jul 14 17:45:28.457661 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.457651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8" event={"ID":"29008bf3-76c0-47da-b097-1281743baf2b","Type":"ContainerDied","Data":"102dcce6a04687d182b6b3e7327a468ac35f24f7cef6c7ed60a8fe8611e712d6"}
Jul 14 17:45:28.457805 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.457676    2580 scope.go:117] "RemoveContainer" containerID="ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd"
Jul 14 17:45:28.459417 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.459327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q" event={"ID":"49a9a556-24a9-46fd-afb9-38b324380a16","Type":"ContainerStarted","Data":"57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13"}
Jul 14 17:45:28.459417 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.459370    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q" event={"ID":"49a9a556-24a9-46fd-afb9-38b324380a16","Type":"ContainerStarted","Data":"040e4015c4708d3bb8f8d59e1424cab5ef0a4ca4cfd93d6498929ee2436e1cb3"}
Jul 14 17:45:28.459614 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.459459    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:28.466916 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.466894    2580 scope.go:117] "RemoveContainer" containerID="f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc"
Jul 14 17:45:28.475374 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.475355    2580 scope.go:117] "RemoveContainer" containerID="ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd"
Jul 14 17:45:28.475632 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:45:28.475614    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd\": container with ID starting with ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd not found: ID does not exist" containerID="ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd"
Jul 14 17:45:28.475684 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.475641    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd"} err="failed to get container status \"ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd\": rpc error: code = NotFound desc = could not find container \"ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd\": container with ID starting with ac5c577444befa70f357bb093c93a741c6606696bc3ef1fa81a55d8e4c7c30cd not found: ID does not exist"
Jul 14 17:45:28.475684 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.475660    2580 scope.go:117] "RemoveContainer" containerID="f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc"
Jul 14 17:45:28.475930 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:45:28.475912    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc\": container with ID starting with f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc not found: ID does not exist" containerID="f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc"
Jul 14 17:45:28.475998 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.475941    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc"} err="failed to get container status \"f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc\": rpc error: code = NotFound desc = could not find container \"f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc\": container with ID starting with f983f3482e2bf1e2012d595f38e0617b06e1c3c9977e1373dae6a4dee930ffdc not found: ID does not exist"
Jul 14 17:45:28.485173 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.485129    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q" podStartSLOduration=2.165772813 podStartE2EDuration="2.485112256s" totalImagesPullingTime="319.339443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:45:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:45:28.483683462 +0000 UTC m=+2415.231283355" watchObservedRunningTime="2026-07-14 17:45:28.485112256 +0000 UTC m=+2415.232712132"
Jul 14 17:45:28.499986 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.499952    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"]
Jul 14 17:45:28.505016 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:28.504993    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-6f8249-openshift-default-69558ff44c-llnr8"]
Jul 14 17:45:29.882083 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:29.882050    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29008bf3-76c0-47da-b097-1281743baf2b" path="/var/lib/kubelet/pods/29008bf3-76c0-47da-b097-1281743baf2b/volumes"
Jul 14 17:45:31.521427 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.521392    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"]
Jul 14 17:45:31.521790 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.521754    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy"
Jul 14 17:45:31.521790 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.521765    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy"
Jul 14 17:45:31.521889 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.521833    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy"
Jul 14 17:45:31.521889 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.521861    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy"
Jul 14 17:45:31.524765 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.524744    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.527378 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.527356    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-df3dc5-openshift-default-dockercfg-4pdcj\""
Jul 14 17:45:31.537288 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.537265    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"]
Jul 14 17:45:31.690275 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.690225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-token\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.690467 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.690292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-podinfo\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.690467 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.690351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-credential-socket\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.690467 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.690378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-data\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.690467 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.690400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3ded19-a744-40f9-893d-9d3880a168a4-istiod-ca-cert\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.690637 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.690519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-socket\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.690637 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.690553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdr72\" (UniqueName: \"kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-kube-api-access-mdr72\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.690637 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.690588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-envoy\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.690637 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.690629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-certs\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.791489 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.791378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-certs\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.791489 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.791425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-token\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.791680 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.791533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-podinfo\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.791680 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.791600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-credential-socket\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.791680 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.791641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-data\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.791680 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.791674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3ded19-a744-40f9-893d-9d3880a168a4-istiod-ca-cert\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.791961 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.791739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-socket\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.791961 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.791763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdr72\" (UniqueName: \"kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-kube-api-access-mdr72\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.791961 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.791915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-certs\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.792140 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.791969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-envoy\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.792140 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.792008    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-credential-socket\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.792140 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.792079    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-data\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.792140 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.792106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-socket\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.792397 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.792376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3ded19-a744-40f9-893d-9d3880a168a4-istiod-ca-cert\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.794233 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.794216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-podinfo\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.794293 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.794216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-envoy\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.802008 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.801986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-token\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.802507 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.802480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdr72\" (UniqueName: \"kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-kube-api-access-mdr72\") pod \"e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") " pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:31.835415 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:31.835378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:32.003059 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:32.002894    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"]
Jul 14 17:45:32.005791 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:45:32.005756    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3ded19_a744_40f9_893d_9d3880a168a4.slice/crio-6516e9c4da6b04a9f988cf2e772a1adf7e0199521908e1c513a08534f8c3546a WatchSource:0}: Error finding container 6516e9c4da6b04a9f988cf2e772a1adf7e0199521908e1c513a08534f8c3546a: Status 404 returned error can't find the container with id 6516e9c4da6b04a9f988cf2e772a1adf7e0199521908e1c513a08534f8c3546a
Jul 14 17:45:32.008200 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:32.008159    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:45:32.008340 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:32.008261    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:45:32.008340 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:32.008308    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:45:32.064074 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:32.064050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:45:32.488201 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:32.488161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" event={"ID":"9c3ded19-a744-40f9-893d-9d3880a168a4","Type":"ContainerStarted","Data":"5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846"}
Jul 14 17:45:32.488401 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:32.488258    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" event={"ID":"9c3ded19-a744-40f9-893d-9d3880a168a4","Type":"ContainerStarted","Data":"6516e9c4da6b04a9f988cf2e772a1adf7e0199521908e1c513a08534f8c3546a"}
Jul 14 17:45:32.513627 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:32.513540    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" podStartSLOduration=1.5135209349999998 podStartE2EDuration="1.513520935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:45:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:45:32.511967188 +0000 UTC m=+2419.259567079" watchObservedRunningTime="2026-07-14 17:45:32.513520935 +0000 UTC m=+2419.261120815"
Jul 14 17:45:32.836283 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:32.836195    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:33.841236 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:33.841157    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:34.472522 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:34.472489    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:45:34.497388 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:34.497356    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:34.498452 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:34.498430    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:44.953571 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:44.953529    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:45:46.559982 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:46.559942    2580 generic.go:356] "Generic (PLEG): container finished" podID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerID="5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846" exitCode=0
Jul 14 17:45:46.560413 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:46.560001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" event={"ID":"9c3ded19-a744-40f9-893d-9d3880a168a4","Type":"ContainerDied","Data":"5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846"}
Jul 14 17:45:46.560469 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:46.560424    2580 scope.go:117] "RemoveContainer" containerID="5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846"
Jul 14 17:45:46.561543 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:46.561511    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:45:46.561615 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:46.561598    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:45:46.561668 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:46.561649    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 14 17:45:47.567267 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:47.567220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" event={"ID":"9c3ded19-a744-40f9-893d-9d3880a168a4","Type":"ContainerStarted","Data":"65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7"}
Jul 14 17:45:47.835989 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:47.835889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:47.836878 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:47.836827    2580 patch_prober.go:28] interesting pod/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 14 17:45:47.836997 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:47.836905    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 14 17:45:48.836770 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:48.836734    2580 patch_prober.go:28] interesting pod/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 14 17:45:48.837195 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:48.836790    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 14 17:45:49.497961 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:49.497931    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:49.836430 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:49.836344    2580 patch_prober.go:28] interesting pod/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 14 17:45:49.836430 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:49.836397    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 14 17:45:50.836386 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:50.836342    2580 patch_prober.go:28] interesting pod/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 14 17:45:50.836912 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:50.836409    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 14 17:45:51.839583 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.839551    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:51.840695 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.840674    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:45:51.952740 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.952707    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"]
Jul 14 17:45:51.953206 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.953190    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy"
Jul 14 17:45:51.953251 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.953209    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29008bf3-76c0-47da-b097-1281743baf2b" containerName="istio-proxy"
Jul 14 17:45:51.956194 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.956178    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:51.958785 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.958765    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-246d80\"/\"default-dockercfg-45tmv\""
Jul 14 17:45:51.959249 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.959233    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-246d80\"/\"auth-test-model-246d80-kserve-self-signed-certs\""
Jul 14 17:45:51.959356 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.959293    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-246d80\"/\"openshift-service-ca.crt\""
Jul 14 17:45:51.959356 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.959338    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-246d80\"/\"kube-root-ca.crt\""
Jul 14 17:45:51.973590 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:51.973559    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"]
Jul 14 17:45:52.003429 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.003389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-home\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.003429 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.003428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5988fe91-cc54-4fa0-b205-a8312adb4275-tls-certs\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.003633 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.003451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8fkp\" (UniqueName: \"kubernetes.io/projected/5988fe91-cc54-4fa0-b205-a8312adb4275-kube-api-access-r8fkp\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.003633 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.003475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-dshm\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.003633 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.003572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-model-cache\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.003633 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.003604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-tmp-dir\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.104244 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.104145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-home\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.104244 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.104185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5988fe91-cc54-4fa0-b205-a8312adb4275-tls-certs\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.104244 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.104203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8fkp\" (UniqueName: \"kubernetes.io/projected/5988fe91-cc54-4fa0-b205-a8312adb4275-kube-api-access-r8fkp\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.104473 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.104345    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-dshm\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.104473 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.104441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-model-cache\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.104540 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.104475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-tmp-dir\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.104634 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.104616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-home\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.104795 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.104769    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-model-cache\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.104946 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.104778    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-tmp-dir\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.106699 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.106679    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-dshm\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.107032 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.107016    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5988fe91-cc54-4fa0-b205-a8312adb4275-tls-certs\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.114703 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.114675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8fkp\" (UniqueName: \"kubernetes.io/projected/5988fe91-cc54-4fa0-b205-a8312adb4275-kube-api-access-r8fkp\") pod \"auth-test-model-246d80-kserve-688b77f7c9-lnw7x\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") " pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.266578 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.266539    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:45:52.432464 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.432425    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"]
Jul 14 17:45:52.444021 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:45:52.443982    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5988fe91_cc54_4fa0_b205_a8312adb4275.slice/crio-12c1c0c5d0926528976a845756b75fe84bebaaaa006981c16a32bc7c1eb68b16 WatchSource:0}: Error finding container 12c1c0c5d0926528976a845756b75fe84bebaaaa006981c16a32bc7c1eb68b16: Status 404 returned error can't find the container with id 12c1c0c5d0926528976a845756b75fe84bebaaaa006981c16a32bc7c1eb68b16
Jul 14 17:45:52.602004 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.601963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x" event={"ID":"5988fe91-cc54-4fa0-b205-a8312adb4275","Type":"ContainerStarted","Data":"24a9dc458824b9c62c2b6f6d869fae5d03f53ae9c767915b54a401a447134b5c"}
Jul 14 17:45:52.602004 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.602009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x" event={"ID":"5988fe91-cc54-4fa0-b205-a8312adb4275","Type":"ContainerStarted","Data":"12c1c0c5d0926528976a845756b75fe84bebaaaa006981c16a32bc7c1eb68b16"}
Jul 14 17:45:52.623152 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:52.623044    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x" podStartSLOduration=1.6230272239999999 podStartE2EDuration="1.623027224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:45:52.622476265 +0000 UTC m=+2439.370076133" watchObservedRunningTime="2026-07-14 17:45:52.623027224 +0000 UTC m=+2439.370627099"
Jul 14 17:45:56.437482 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:45:56.437450    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:46:02.266690 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:46:02.266651    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:46:02.267120 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:46:02.266707    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:46:02.279713 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:46:02.279684    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:46:02.663321 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:46:02.663235    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:46:23.337003 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:46:23.336955    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:46:38.283770 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:46:38.283729    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:46:56.481615 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:46:56.481571    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:47:13.137500 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:47:13.137413    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:47:31.054879 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:47:31.054817    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:47:45.561299 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:47:45.561263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:48:05.244704 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:05.244667    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:48:21.541879 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:21.541830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:48:26.234981 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:26.234946    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"]
Jul 14 17:48:39.045468 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:39.045433    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:48:41.256009 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.255956    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x" podUID="5988fe91-cc54-4fa0-b205-a8312adb4275" containerName="main" containerID="cri-o://24a9dc458824b9c62c2b6f6d869fae5d03f53ae9c767915b54a401a447134b5c" gracePeriod=45
Jul 14 17:48:41.419647 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.419616    2580 generic.go:356] "Generic (PLEG): container finished" podID="5988fe91-cc54-4fa0-b205-a8312adb4275" containerID="24a9dc458824b9c62c2b6f6d869fae5d03f53ae9c767915b54a401a447134b5c" exitCode=0
Jul 14 17:48:41.419818 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.419690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x" event={"ID":"5988fe91-cc54-4fa0-b205-a8312adb4275","Type":"ContainerDied","Data":"24a9dc458824b9c62c2b6f6d869fae5d03f53ae9c767915b54a401a447134b5c"}
Jul 14 17:48:41.511075 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.510999    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:48:41.576170 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.576136    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8fkp\" (UniqueName: \"kubernetes.io/projected/5988fe91-cc54-4fa0-b205-a8312adb4275-kube-api-access-r8fkp\") pod \"5988fe91-cc54-4fa0-b205-a8312adb4275\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") "
Jul 14 17:48:41.576374 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.576198    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-home\") pod \"5988fe91-cc54-4fa0-b205-a8312adb4275\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") "
Jul 14 17:48:41.576374 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.576249    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5988fe91-cc54-4fa0-b205-a8312adb4275-tls-certs\") pod \"5988fe91-cc54-4fa0-b205-a8312adb4275\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") "
Jul 14 17:48:41.576374 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.576271    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-tmp-dir\") pod \"5988fe91-cc54-4fa0-b205-a8312adb4275\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") "
Jul 14 17:48:41.576374 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.576313    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-dshm\") pod \"5988fe91-cc54-4fa0-b205-a8312adb4275\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") "
Jul 14 17:48:41.576374 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.576328    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-model-cache\") pod \"5988fe91-cc54-4fa0-b205-a8312adb4275\" (UID: \"5988fe91-cc54-4fa0-b205-a8312adb4275\") "
Jul 14 17:48:41.576630 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.576479    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-home" (OuterVolumeSpecName: "home") pod "5988fe91-cc54-4fa0-b205-a8312adb4275" (UID: "5988fe91-cc54-4fa0-b205-a8312adb4275"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:48:41.576630 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.576556    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5988fe91-cc54-4fa0-b205-a8312adb4275" (UID: "5988fe91-cc54-4fa0-b205-a8312adb4275"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:48:41.576697 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.576680    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-model-cache" (OuterVolumeSpecName: "model-cache") pod "5988fe91-cc54-4fa0-b205-a8312adb4275" (UID: "5988fe91-cc54-4fa0-b205-a8312adb4275"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:48:41.578674 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.578646    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5988fe91-cc54-4fa0-b205-a8312adb4275-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5988fe91-cc54-4fa0-b205-a8312adb4275" (UID: "5988fe91-cc54-4fa0-b205-a8312adb4275"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:48:41.578674 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.578665    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5988fe91-cc54-4fa0-b205-a8312adb4275-kube-api-access-r8fkp" (OuterVolumeSpecName: "kube-api-access-r8fkp") pod "5988fe91-cc54-4fa0-b205-a8312adb4275" (UID: "5988fe91-cc54-4fa0-b205-a8312adb4275"). InnerVolumeSpecName "kube-api-access-r8fkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:48:41.578818 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.578715    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-dshm" (OuterVolumeSpecName: "dshm") pod "5988fe91-cc54-4fa0-b205-a8312adb4275" (UID: "5988fe91-cc54-4fa0-b205-a8312adb4275"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:48:41.677478 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.677438    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-tmp-dir\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:48:41.677478 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.677472    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-dshm\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:48:41.677478 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.677481    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-model-cache\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:48:41.677716 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.677491    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8fkp\" (UniqueName: \"kubernetes.io/projected/5988fe91-cc54-4fa0-b205-a8312adb4275-kube-api-access-r8fkp\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:48:41.677716 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.677500    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5988fe91-cc54-4fa0-b205-a8312adb4275-home\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:48:41.677716 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:41.677508    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5988fe91-cc54-4fa0-b205-a8312adb4275-tls-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:48:42.426200 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:42.426169    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"
Jul 14 17:48:42.426702 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:42.426174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x" event={"ID":"5988fe91-cc54-4fa0-b205-a8312adb4275","Type":"ContainerDied","Data":"12c1c0c5d0926528976a845756b75fe84bebaaaa006981c16a32bc7c1eb68b16"}
Jul 14 17:48:42.426702 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:42.426298    2580 scope.go:117] "RemoveContainer" containerID="24a9dc458824b9c62c2b6f6d869fae5d03f53ae9c767915b54a401a447134b5c"
Jul 14 17:48:42.457319 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:42.457285    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"]
Jul 14 17:48:42.459180 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:42.459156    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/auth-test-model-246d80-kserve-688b77f7c9-lnw7x"]
Jul 14 17:48:43.882608 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:43.882569    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5988fe91-cc54-4fa0-b205-a8312adb4275" path="/var/lib/kubelet/pods/5988fe91-cc54-4fa0-b205-a8312adb4275/volumes"
Jul 14 17:48:47.643575 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:48:47.643540    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:49:06.331877 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.331822    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"]
Jul 14 17:49:06.332413 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.332392    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5988fe91-cc54-4fa0-b205-a8312adb4275" containerName="main"
Jul 14 17:49:06.332492 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.332417    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5988fe91-cc54-4fa0-b205-a8312adb4275" containerName="main"
Jul 14 17:49:06.332549 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.332518    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5988fe91-cc54-4fa0-b205-a8312adb4275" containerName="main"
Jul 14 17:49:06.335637 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.335613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.339152 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.339119    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"openshift-service-ca.crt\""
Jul 14 17:49:06.340290 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.340267    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"sub-test-model-df3dc5-kserve-self-signed-certs\""
Jul 14 17:49:06.340418 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.340310    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"default-dockercfg-vhsbb\""
Jul 14 17:49:06.340488 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.340317    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"kube-root-ca.crt\""
Jul 14 17:49:06.347611 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.347587    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"]
Jul 14 17:49:06.418161 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.418120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdbd8462-eb88-4760-9351-804f4a882141-tls-certs\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.418339 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.418181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdzp4\" (UniqueName: \"kubernetes.io/projected/fdbd8462-eb88-4760-9351-804f4a882141-kube-api-access-zdzp4\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.418339 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.418310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-home\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.418483 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.418383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-model-cache\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.418483 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.418418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-dshm\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.418552 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.418508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-tmp-dir\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.519637 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.519599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-model-cache\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.519637 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.519653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-dshm\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.520094 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.519701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-tmp-dir\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.520094 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.519744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdbd8462-eb88-4760-9351-804f4a882141-tls-certs\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.520094 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.519778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdzp4\" (UniqueName: \"kubernetes.io/projected/fdbd8462-eb88-4760-9351-804f4a882141-kube-api-access-zdzp4\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.520094 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.519869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-home\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.520370 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.520190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-model-cache\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.520561 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.520479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-tmp-dir\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.520657 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.520615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-home\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.523418 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.523387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-dshm\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.523764 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.523743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdbd8462-eb88-4760-9351-804f4a882141-tls-certs\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.540163 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.540128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdzp4\" (UniqueName: \"kubernetes.io/projected/fdbd8462-eb88-4760-9351-804f4a882141-kube-api-access-zdzp4\") pod \"sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") " pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.647616 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.647505    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:06.795326 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:06.795297    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"]
Jul 14 17:49:06.797563 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:49:06.797524    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdbd8462_eb88_4760_9351_804f4a882141.slice/crio-e9a9872bafda607f73a76d03fc1ff2a07123d8fb040c64bc022330b5fb295e2b WatchSource:0}: Error finding container e9a9872bafda607f73a76d03fc1ff2a07123d8fb040c64bc022330b5fb295e2b: Status 404 returned error can't find the container with id e9a9872bafda607f73a76d03fc1ff2a07123d8fb040c64bc022330b5fb295e2b
Jul 14 17:49:07.552428 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:07.552391    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t" event={"ID":"fdbd8462-eb88-4760-9351-804f4a882141","Type":"ContainerStarted","Data":"9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd"}
Jul 14 17:49:07.552428 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:07.552435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t" event={"ID":"fdbd8462-eb88-4760-9351-804f4a882141","Type":"ContainerStarted","Data":"e9a9872bafda607f73a76d03fc1ff2a07123d8fb040c64bc022330b5fb295e2b"}
Jul 14 17:49:07.575162 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:07.575091    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t" podStartSLOduration=1.5750708119999999 podStartE2EDuration="1.575070812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:49:07.574720797 +0000 UTC m=+2634.322320674" watchObservedRunningTime="2026-07-14 17:49:07.575070812 +0000 UTC m=+2634.322670689"
Jul 14 17:49:10.465560 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:10.465524    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:49:16.647708 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:16.647662    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:16.648141 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:16.647724    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:16.660641 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:16.660616    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:17.615723 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:17.615687    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:49:35.241582 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:35.241544    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:49:51.963358 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:51.963320    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"]
Jul 14 17:49:55.442277 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:49:55.442238    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:50:06.984624 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:06.984522    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t" podUID="fdbd8462-eb88-4760-9351-804f4a882141" containerName="main" containerID="cri-o://9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd" gracePeriod=45
Jul 14 17:50:07.241467 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.241399    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:50:07.282488 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.282453    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-home\") pod \"fdbd8462-eb88-4760-9351-804f4a882141\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") "
Jul 14 17:50:07.282695 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.282504    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-dshm\") pod \"fdbd8462-eb88-4760-9351-804f4a882141\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") "
Jul 14 17:50:07.282695 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.282552    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdzp4\" (UniqueName: \"kubernetes.io/projected/fdbd8462-eb88-4760-9351-804f4a882141-kube-api-access-zdzp4\") pod \"fdbd8462-eb88-4760-9351-804f4a882141\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") "
Jul 14 17:50:07.282695 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.282596    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-model-cache\") pod \"fdbd8462-eb88-4760-9351-804f4a882141\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") "
Jul 14 17:50:07.282695 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.282622    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-tmp-dir\") pod \"fdbd8462-eb88-4760-9351-804f4a882141\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") "
Jul 14 17:50:07.282695 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.282670    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdbd8462-eb88-4760-9351-804f4a882141-tls-certs\") pod \"fdbd8462-eb88-4760-9351-804f4a882141\" (UID: \"fdbd8462-eb88-4760-9351-804f4a882141\") "
Jul 14 17:50:07.283023 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.282740    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-home" (OuterVolumeSpecName: "home") pod "fdbd8462-eb88-4760-9351-804f4a882141" (UID: "fdbd8462-eb88-4760-9351-804f4a882141"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:50:07.283023 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.282888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-model-cache" (OuterVolumeSpecName: "model-cache") pod "fdbd8462-eb88-4760-9351-804f4a882141" (UID: "fdbd8462-eb88-4760-9351-804f4a882141"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:50:07.283023 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.282963    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fdbd8462-eb88-4760-9351-804f4a882141" (UID: "fdbd8462-eb88-4760-9351-804f4a882141"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:50:07.283180 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.283043    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-tmp-dir\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:50:07.283180 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.283060    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-home\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:50:07.283180 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.283068    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-model-cache\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:50:07.285112 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.285077    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-dshm" (OuterVolumeSpecName: "dshm") pod "fdbd8462-eb88-4760-9351-804f4a882141" (UID: "fdbd8462-eb88-4760-9351-804f4a882141"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:50:07.285112 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.285095    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdbd8462-eb88-4760-9351-804f4a882141-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fdbd8462-eb88-4760-9351-804f4a882141" (UID: "fdbd8462-eb88-4760-9351-804f4a882141"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:50:07.285271 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.285114    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdbd8462-eb88-4760-9351-804f4a882141-kube-api-access-zdzp4" (OuterVolumeSpecName: "kube-api-access-zdzp4") pod "fdbd8462-eb88-4760-9351-804f4a882141" (UID: "fdbd8462-eb88-4760-9351-804f4a882141"). InnerVolumeSpecName "kube-api-access-zdzp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:50:07.384481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.384453    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdbd8462-eb88-4760-9351-804f4a882141-dshm\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:50:07.384481 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.384481    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdzp4\" (UniqueName: \"kubernetes.io/projected/fdbd8462-eb88-4760-9351-804f4a882141-kube-api-access-zdzp4\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:50:07.384698 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.384493    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdbd8462-eb88-4760-9351-804f4a882141-tls-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:50:07.853030 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.852996    2580 generic.go:356] "Generic (PLEG): container finished" podID="fdbd8462-eb88-4760-9351-804f4a882141" containerID="9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd" exitCode=0
Jul 14 17:50:07.853231 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.853038    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"
Jul 14 17:50:07.853231 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.853060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t" event={"ID":"fdbd8462-eb88-4760-9351-804f4a882141","Type":"ContainerDied","Data":"9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd"}
Jul 14 17:50:07.853231 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.853091    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t" event={"ID":"fdbd8462-eb88-4760-9351-804f4a882141","Type":"ContainerDied","Data":"e9a9872bafda607f73a76d03fc1ff2a07123d8fb040c64bc022330b5fb295e2b"}
Jul 14 17:50:07.853231 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.853108    2580 scope.go:117] "RemoveContainer" containerID="9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd"
Jul 14 17:50:07.862438 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.862419    2580 scope.go:117] "RemoveContainer" containerID="9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd"
Jul 14 17:50:07.862710 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:50:07.862692    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd\": container with ID starting with 9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd not found: ID does not exist" containerID="9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd"
Jul 14 17:50:07.862763 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.862721    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd"} err="failed to get container status \"9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd\": rpc error: code = NotFound desc = could not find container \"9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd\": container with ID starting with 9d9507fce37ca214f172ba367e3495111b77e2bc5e24881736fafc70798d25cd not found: ID does not exist"
Jul 14 17:50:07.876109 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.876081    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"]
Jul 14 17:50:07.885239 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:07.885201    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/sub-test-model-df3dc5-kserve-69ff68f6d5-kft6t"]
Jul 14 17:50:09.881762 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:09.881725    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdbd8462-eb88-4760-9351-804f4a882141" path="/var/lib/kubelet/pods/fdbd8462-eb88-4760-9351-804f4a882141/volumes"
Jul 14 17:50:10.545155 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:10.545120    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:50:13.904751 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:13.904723    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:50:13.912958 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:13.912937    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59wt_b90c2ddb-a4ec-4f83-8a92-6f629b633810/ovn-acl-logging/0.log"
Jul 14 17:50:40.669755 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.669721    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"]
Jul 14 17:50:40.670210 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.670159    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdbd8462-eb88-4760-9351-804f4a882141" containerName="main"
Jul 14 17:50:40.670210 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.670171    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdbd8462-eb88-4760-9351-804f4a882141" containerName="main"
Jul 14 17:50:40.670287 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.670229    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdbd8462-eb88-4760-9351-804f4a882141" containerName="main"
Jul 14 17:50:40.673353 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.673337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.677236 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.677209    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"default-dockercfg-vhsbb\""
Jul 14 17:50:40.677389 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.677208    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"openshift-service-ca.crt\""
Jul 14 17:50:40.677389 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.677211    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"kube-root-ca.crt\""
Jul 14 17:50:40.677389 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.677210    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"rate-test-model-df3dc5-kserve-self-signed-certs\""
Jul 14 17:50:40.689385 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.689359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"]
Jul 14 17:50:40.703738 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.703698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-model-cache\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.703946 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.703796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tls-certs\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.703946 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.703907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tmp-dir\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.704082 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.703955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-dshm\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.704082 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.704063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfhvx\" (UniqueName: \"kubernetes.io/projected/c54fca8c-858a-42ad-b778-d0b2525e2f5a-kube-api-access-lfhvx\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.704168 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.704103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-home\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.805032 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.804996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-model-cache\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.805232 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.805041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tls-certs\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.805232 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.805078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tmp-dir\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.805232 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.805104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-dshm\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.805232 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.805160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfhvx\" (UniqueName: \"kubernetes.io/projected/c54fca8c-858a-42ad-b778-d0b2525e2f5a-kube-api-access-lfhvx\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.805450 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.805293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-home\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.805509 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.805450    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-model-cache\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.805565 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.805502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tmp-dir\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.805608 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.805589    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-home\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.807545 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.807521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-dshm\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.807801 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.807782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tls-certs\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.814464 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.814441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfhvx\" (UniqueName: \"kubernetes.io/projected/c54fca8c-858a-42ad-b778-d0b2525e2f5a-kube-api-access-lfhvx\") pod \"rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") " pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:40.982388 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:40.982355    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:41.136476 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:41.136447    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"]
Jul 14 17:50:41.138242 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:50:41.138212    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc54fca8c_858a_42ad_b778_d0b2525e2f5a.slice/crio-034b30020de20f34580b882cb09b4eb84cd8c4333d3f7c29d6c0d98fa85c9f35 WatchSource:0}: Error finding container 034b30020de20f34580b882cb09b4eb84cd8c4333d3f7c29d6c0d98fa85c9f35: Status 404 returned error can't find the container with id 034b30020de20f34580b882cb09b4eb84cd8c4333d3f7c29d6c0d98fa85c9f35
Jul 14 17:50:42.024627 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:42.024592    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m" event={"ID":"c54fca8c-858a-42ad-b778-d0b2525e2f5a","Type":"ContainerStarted","Data":"f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc"}
Jul 14 17:50:42.025048 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:42.024636    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m" event={"ID":"c54fca8c-858a-42ad-b778-d0b2525e2f5a","Type":"ContainerStarted","Data":"034b30020de20f34580b882cb09b4eb84cd8c4333d3f7c29d6c0d98fa85c9f35"}
Jul 14 17:50:42.044438 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:42.044382    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m" podStartSLOduration=2.044364916 podStartE2EDuration="2.044364916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:50:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:50:42.043259219 +0000 UTC m=+2728.790859133" watchObservedRunningTime="2026-07-14 17:50:42.044364916 +0000 UTC m=+2728.791964812"
Jul 14 17:50:45.044320 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:45.044284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:50:50.983190 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:50.983144    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:50.985558 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:50.983209    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:50.995878 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:50.995831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:50:51.080299 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:50:51.080266    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:51:00.369269 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:00.369233    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:51:16.564429 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:16.564383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:51:24.323710 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:24.323675    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"]
Jul 14 17:51:25.310857 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:25.310792    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7335-140017961548464"]
Jul 14 17:51:25.314450 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:25.314422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961548464"
Jul 14 17:51:25.317098 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:25.317073    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-8744v\""
Jul 14 17:51:25.322651 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:25.322624    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961548464"]
Jul 14 17:51:25.421987 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:25.421949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjrrd\" (UniqueName: \"kubernetes.io/projected/0dbf750c-38c1-4ebb-9b4e-b2e841287bb3-kube-api-access-gjrrd\") pod \"test-curl-7335-140017961548464\" (UID: \"0dbf750c-38c1-4ebb-9b4e-b2e841287bb3\") " pod="openshift-ingress/test-curl-7335-140017961548464"
Jul 14 17:51:25.522760 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:25.522714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjrrd\" (UniqueName: \"kubernetes.io/projected/0dbf750c-38c1-4ebb-9b4e-b2e841287bb3-kube-api-access-gjrrd\") pod \"test-curl-7335-140017961548464\" (UID: \"0dbf750c-38c1-4ebb-9b4e-b2e841287bb3\") " pod="openshift-ingress/test-curl-7335-140017961548464"
Jul 14 17:51:25.532468 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:25.532441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjrrd\" (UniqueName: \"kubernetes.io/projected/0dbf750c-38c1-4ebb-9b4e-b2e841287bb3-kube-api-access-gjrrd\") pod \"test-curl-7335-140017961548464\" (UID: \"0dbf750c-38c1-4ebb-9b4e-b2e841287bb3\") " pod="openshift-ingress/test-curl-7335-140017961548464"
Jul 14 17:51:25.625492 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:25.625398    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961548464"
Jul 14 17:51:25.770492 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:25.770469    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961548464"]
Jul 14 17:51:25.772668 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:51:25.772641    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0dbf750c_38c1_4ebb_9b4e_b2e841287bb3.slice/crio-503fb1cf4720cdb4a5e71d0644124c76ca413ea3ba293eebb1a84c6c2c4cbb77 WatchSource:0}: Error finding container 503fb1cf4720cdb4a5e71d0644124c76ca413ea3ba293eebb1a84c6c2c4cbb77: Status 404 returned error can't find the container with id 503fb1cf4720cdb4a5e71d0644124c76ca413ea3ba293eebb1a84c6c2c4cbb77
Jul 14 17:51:25.774129 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:25.774108    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 17:51:26.248685 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:26.248646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961548464" event={"ID":"0dbf750c-38c1-4ebb-9b4e-b2e841287bb3","Type":"ContainerStarted","Data":"503fb1cf4720cdb4a5e71d0644124c76ca413ea3ba293eebb1a84c6c2c4cbb77"}
Jul 14 17:51:27.255226 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:27.255193    2580 generic.go:356] "Generic (PLEG): container finished" podID="0dbf750c-38c1-4ebb-9b4e-b2e841287bb3" containerID="28c99d32e9039473b488ce497b18cafab0ceccbb2f98185f1f112177063ec5c6" exitCode=0
Jul 14 17:51:27.255606 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:27.255279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961548464" event={"ID":"0dbf750c-38c1-4ebb-9b4e-b2e841287bb3","Type":"ContainerDied","Data":"28c99d32e9039473b488ce497b18cafab0ceccbb2f98185f1f112177063ec5c6"}
Jul 14 17:51:28.408739 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:28.408707    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961548464"
Jul 14 17:51:28.451146 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:28.451119    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7335-140017961548464_0dbf750c-38c1-4ebb-9b4e-b2e841287bb3/test-curl-7335-140017961548464/0.log"
Jul 14 17:51:28.474593 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:28.474562    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7335-140017961548464"]
Jul 14 17:51:28.477884 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:28.477839    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7335-140017961548464"]
Jul 14 17:51:28.549049 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:28.548959    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjrrd\" (UniqueName: \"kubernetes.io/projected/0dbf750c-38c1-4ebb-9b4e-b2e841287bb3-kube-api-access-gjrrd\") pod \"0dbf750c-38c1-4ebb-9b4e-b2e841287bb3\" (UID: \"0dbf750c-38c1-4ebb-9b4e-b2e841287bb3\") "
Jul 14 17:51:28.551208 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:28.551187    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dbf750c-38c1-4ebb-9b4e-b2e841287bb3-kube-api-access-gjrrd" (OuterVolumeSpecName: "kube-api-access-gjrrd") pod "0dbf750c-38c1-4ebb-9b4e-b2e841287bb3" (UID: "0dbf750c-38c1-4ebb-9b4e-b2e841287bb3"). InnerVolumeSpecName "kube-api-access-gjrrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:51:28.650379 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:28.650336    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjrrd\" (UniqueName: \"kubernetes.io/projected/0dbf750c-38c1-4ebb-9b4e-b2e841287bb3-kube-api-access-gjrrd\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:51:29.265925 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:29.265892    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="503fb1cf4720cdb4a5e71d0644124c76ca413ea3ba293eebb1a84c6c2c4cbb77"
Jul 14 17:51:29.265925 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:29.265926    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961548464"
Jul 14 17:51:29.275700 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:51:29.275672    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7335-140017961548464\" is forbidden: User \"system:node:ip-10-0-137-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-137-134.ec2.internal' and this object" podUID="0dbf750c-38c1-4ebb-9b4e-b2e841287bb3" pod="openshift-ingress/test-curl-7335-140017961548464"
Jul 14 17:51:29.882893 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:29.882857    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dbf750c-38c1-4ebb-9b4e-b2e841287bb3" path="/var/lib/kubelet/pods/0dbf750c-38c1-4ebb-9b4e-b2e841287bb3/volumes"
Jul 14 17:51:30.937332 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:30.937295    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:51:39.344425 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.344237    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m" podUID="c54fca8c-858a-42ad-b778-d0b2525e2f5a" containerName="main" containerID="cri-o://f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc" gracePeriod=45
Jul 14 17:51:39.604276 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.604213    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:51:39.652226 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.652197    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-dshm\") pod \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") "
Jul 14 17:51:39.652226 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.652226    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tmp-dir\") pod \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") "
Jul 14 17:51:39.652451 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.652278    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfhvx\" (UniqueName: \"kubernetes.io/projected/c54fca8c-858a-42ad-b778-d0b2525e2f5a-kube-api-access-lfhvx\") pod \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") "
Jul 14 17:51:39.652451 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.652365    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-home\") pod \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") "
Jul 14 17:51:39.652451 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.652417    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tls-certs\") pod \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") "
Jul 14 17:51:39.652451 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.652444    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-model-cache\") pod \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\" (UID: \"c54fca8c-858a-42ad-b778-d0b2525e2f5a\") "
Jul 14 17:51:39.652664 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.652538    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c54fca8c-858a-42ad-b778-d0b2525e2f5a" (UID: "c54fca8c-858a-42ad-b778-d0b2525e2f5a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:51:39.652779 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.652746    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tmp-dir\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:51:39.652779 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.652770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-model-cache" (OuterVolumeSpecName: "model-cache") pod "c54fca8c-858a-42ad-b778-d0b2525e2f5a" (UID: "c54fca8c-858a-42ad-b778-d0b2525e2f5a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:51:39.653036 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.652790    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-home" (OuterVolumeSpecName: "home") pod "c54fca8c-858a-42ad-b778-d0b2525e2f5a" (UID: "c54fca8c-858a-42ad-b778-d0b2525e2f5a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:51:39.654577 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.654546    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-dshm" (OuterVolumeSpecName: "dshm") pod "c54fca8c-858a-42ad-b778-d0b2525e2f5a" (UID: "c54fca8c-858a-42ad-b778-d0b2525e2f5a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:51:39.654759 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.654737    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c54fca8c-858a-42ad-b778-d0b2525e2f5a" (UID: "c54fca8c-858a-42ad-b778-d0b2525e2f5a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:51:39.654913 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.654891    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c54fca8c-858a-42ad-b778-d0b2525e2f5a-kube-api-access-lfhvx" (OuterVolumeSpecName: "kube-api-access-lfhvx") pod "c54fca8c-858a-42ad-b778-d0b2525e2f5a" (UID: "c54fca8c-858a-42ad-b778-d0b2525e2f5a"). InnerVolumeSpecName "kube-api-access-lfhvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:51:39.753349 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.753313    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c54fca8c-858a-42ad-b778-d0b2525e2f5a-tls-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:51:39.753349 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.753346    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-model-cache\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:51:39.753349 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.753355    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-dshm\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:51:39.753349 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.753365    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfhvx\" (UniqueName: \"kubernetes.io/projected/c54fca8c-858a-42ad-b778-d0b2525e2f5a-kube-api-access-lfhvx\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:51:39.753636 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:39.753374    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c54fca8c-858a-42ad-b778-d0b2525e2f5a-home\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:51:40.319408 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:40.319375    2580 generic.go:356] "Generic (PLEG): container finished" podID="c54fca8c-858a-42ad-b778-d0b2525e2f5a" containerID="f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc" exitCode=0
Jul 14 17:51:40.319691 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:40.319423    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m" event={"ID":"c54fca8c-858a-42ad-b778-d0b2525e2f5a","Type":"ContainerDied","Data":"f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc"}
Jul 14 17:51:40.319691 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:40.319428    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"
Jul 14 17:51:40.319691 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:40.319454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m" event={"ID":"c54fca8c-858a-42ad-b778-d0b2525e2f5a","Type":"ContainerDied","Data":"034b30020de20f34580b882cb09b4eb84cd8c4333d3f7c29d6c0d98fa85c9f35"}
Jul 14 17:51:40.319691 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:40.319470    2580 scope.go:117] "RemoveContainer" containerID="f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc"
Jul 14 17:51:40.328568 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:40.328549    2580 scope.go:117] "RemoveContainer" containerID="f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc"
Jul 14 17:51:40.328856 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:51:40.328821    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc\": container with ID starting with f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc not found: ID does not exist" containerID="f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc"
Jul 14 17:51:40.328921 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:40.328876    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc"} err="failed to get container status \"f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc\": rpc error: code = NotFound desc = could not find container \"f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc\": container with ID starting with f92bb6b0e94cfc660e114e76ee6dbe49d92633a4051a862ee3f13a2e090f91bc not found: ID does not exist"
Jul 14 17:51:40.349103 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:40.349065    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"]
Jul 14 17:51:40.367587 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:40.367555    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/rate-test-model-df3dc5-kserve-645ccd6c6c-djs2m"]
Jul 14 17:51:41.882349 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:41.882316    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c54fca8c-858a-42ad-b778-d0b2525e2f5a" path="/var/lib/kubelet/pods/c54fca8c-858a-42ad-b778-d0b2525e2f5a/volumes"
Jul 14 17:51:54.273487 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.273454    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"]
Jul 14 17:51:54.274002 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.273704    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q" podUID="49a9a556-24a9-46fd-afb9-38b324380a16" containerName="maas-api" containerID="cri-o://57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13" gracePeriod=30
Jul 14 17:51:54.532009 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.531933    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:51:54.588941 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.588913    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a9a556-24a9-46fd-afb9-38b324380a16-maas-api-tls\") pod \"49a9a556-24a9-46fd-afb9-38b324380a16\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") "
Jul 14 17:51:54.588941 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.588947    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a9a556-24a9-46fd-afb9-38b324380a16-openshift-service-ca\") pod \"49a9a556-24a9-46fd-afb9-38b324380a16\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") "
Jul 14 17:51:54.589213 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.589011    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r7f4\" (UniqueName: \"kubernetes.io/projected/49a9a556-24a9-46fd-afb9-38b324380a16-kube-api-access-4r7f4\") pod \"49a9a556-24a9-46fd-afb9-38b324380a16\" (UID: \"49a9a556-24a9-46fd-afb9-38b324380a16\") "
Jul 14 17:51:54.589397 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.589372    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49a9a556-24a9-46fd-afb9-38b324380a16-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "49a9a556-24a9-46fd-afb9-38b324380a16" (UID: "49a9a556-24a9-46fd-afb9-38b324380a16"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:51:54.591109 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.591085    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49a9a556-24a9-46fd-afb9-38b324380a16-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "49a9a556-24a9-46fd-afb9-38b324380a16" (UID: "49a9a556-24a9-46fd-afb9-38b324380a16"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 17:51:54.591202 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.591156    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49a9a556-24a9-46fd-afb9-38b324380a16-kube-api-access-4r7f4" (OuterVolumeSpecName: "kube-api-access-4r7f4") pod "49a9a556-24a9-46fd-afb9-38b324380a16" (UID: "49a9a556-24a9-46fd-afb9-38b324380a16"). InnerVolumeSpecName "kube-api-access-4r7f4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:51:54.690444 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.690404    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a9a556-24a9-46fd-afb9-38b324380a16-maas-api-tls\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:51:54.690444 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.690436    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a9a556-24a9-46fd-afb9-38b324380a16-openshift-service-ca\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:51:54.690444 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:54.690449    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r7f4\" (UniqueName: \"kubernetes.io/projected/49a9a556-24a9-46fd-afb9-38b324380a16-kube-api-access-4r7f4\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:51:55.393662 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:55.393624    2580 generic.go:356] "Generic (PLEG): container finished" podID="49a9a556-24a9-46fd-afb9-38b324380a16" containerID="57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13" exitCode=0
Jul 14 17:51:55.394174 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:55.393686    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"
Jul 14 17:51:55.394174 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:55.393715    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q" event={"ID":"49a9a556-24a9-46fd-afb9-38b324380a16","Type":"ContainerDied","Data":"57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13"}
Jul 14 17:51:55.394174 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:55.393772    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q" event={"ID":"49a9a556-24a9-46fd-afb9-38b324380a16","Type":"ContainerDied","Data":"040e4015c4708d3bb8f8d59e1424cab5ef0a4ca4cfd93d6498929ee2436e1cb3"}
Jul 14 17:51:55.394174 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:55.393797    2580 scope.go:117] "RemoveContainer" containerID="57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13"
Jul 14 17:51:55.403578 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:55.403561    2580 scope.go:117] "RemoveContainer" containerID="57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13"
Jul 14 17:51:55.403820 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:51:55.403803    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13\": container with ID starting with 57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13 not found: ID does not exist" containerID="57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13"
Jul 14 17:51:55.403922 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:55.403832    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13"} err="failed to get container status \"57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13\": rpc error: code = NotFound desc = could not find container \"57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13\": container with ID starting with 57d6793630cca8dfd4d6ed5470da2f6e9afd156c4f885bb7ac33f8789e189e13 not found: ID does not exist"
Jul 14 17:51:55.417496 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:55.417469    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"]
Jul 14 17:51:55.421427 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:55.421409    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q"]
Jul 14 17:51:55.882677 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:55.882641    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49a9a556-24a9-46fd-afb9-38b324380a16" path="/var/lib/kubelet/pods/49a9a556-24a9-46fd-afb9-38b324380a16/volumes"
Jul 14 17:51:57.466951 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:57.466907    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-246d80-59d65cb54f-5rj4q" podUID="49a9a556-24a9-46fd-afb9-38b324380a16" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.56:8443/health\": context deadline exceeded"
Jul 14 17:51:57.855515 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:51:57.855436    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:52:24.042939 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:24.042896    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-77fzw"]
Jul 14 17:52:35.935343 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:35.935309    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"]
Jul 14 17:52:35.935782 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:35.935560    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy" containerID="cri-o://65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7" gracePeriod=30
Jul 14 17:52:36.842328 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:36.842289    2580 patch_prober.go:28] interesting pod/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 14 17:52:36.842491 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:36.842356    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 17:52:41.202377 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.202309    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:52:41.313675 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.313640    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3ded19-a744-40f9-893d-9d3880a168a4-istiod-ca-cert\") pod \"9c3ded19-a744-40f9-893d-9d3880a168a4\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") "
Jul 14 17:52:41.313675 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.313677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-envoy\") pod \"9c3ded19-a744-40f9-893d-9d3880a168a4\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") "
Jul 14 17:52:41.313934 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.313718    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-token\") pod \"9c3ded19-a744-40f9-893d-9d3880a168a4\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") "
Jul 14 17:52:41.313934 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.313751    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-podinfo\") pod \"9c3ded19-a744-40f9-893d-9d3880a168a4\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") "
Jul 14 17:52:41.313934 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.313771    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-certs\") pod \"9c3ded19-a744-40f9-893d-9d3880a168a4\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") "
Jul 14 17:52:41.313934 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.313830    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-credential-socket\") pod \"9c3ded19-a744-40f9-893d-9d3880a168a4\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") "
Jul 14 17:52:41.313934 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.313892    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-data\") pod \"9c3ded19-a744-40f9-893d-9d3880a168a4\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") "
Jul 14 17:52:41.313934 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.313914    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-socket\") pod \"9c3ded19-a744-40f9-893d-9d3880a168a4\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") "
Jul 14 17:52:41.314226 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.313977    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdr72\" (UniqueName: \"kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-kube-api-access-mdr72\") pod \"9c3ded19-a744-40f9-893d-9d3880a168a4\" (UID: \"9c3ded19-a744-40f9-893d-9d3880a168a4\") "
Jul 14 17:52:41.314226 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.314158    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9c3ded19-a744-40f9-893d-9d3880a168a4" (UID: "9c3ded19-a744-40f9-893d-9d3880a168a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:52:41.314226 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.314175    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9c3ded19-a744-40f9-893d-9d3880a168a4" (UID: "9c3ded19-a744-40f9-893d-9d3880a168a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:52:41.314393 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.314293    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-certs\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:52:41.314393 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.314316    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-credential-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:52:41.314393 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.314380    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9c3ded19-a744-40f9-893d-9d3880a168a4" (UID: "9c3ded19-a744-40f9-893d-9d3880a168a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:52:41.314824 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.314793    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c3ded19-a744-40f9-893d-9d3880a168a4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9c3ded19-a744-40f9-893d-9d3880a168a4" (UID: "9c3ded19-a744-40f9-893d-9d3880a168a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 17:52:41.315191 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.315157    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-data" (OuterVolumeSpecName: "istio-data") pod "9c3ded19-a744-40f9-893d-9d3880a168a4" (UID: "9c3ded19-a744-40f9-893d-9d3880a168a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:52:41.316231 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.316204    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-token" (OuterVolumeSpecName: "istio-token") pod "9c3ded19-a744-40f9-893d-9d3880a168a4" (UID: "9c3ded19-a744-40f9-893d-9d3880a168a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:52:41.316326 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.316241    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9c3ded19-a744-40f9-893d-9d3880a168a4" (UID: "9c3ded19-a744-40f9-893d-9d3880a168a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 17:52:41.316326 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.316270    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9c3ded19-a744-40f9-893d-9d3880a168a4" (UID: "9c3ded19-a744-40f9-893d-9d3880a168a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 17:52:41.316326 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.316320    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-kube-api-access-mdr72" (OuterVolumeSpecName: "kube-api-access-mdr72") pod "9c3ded19-a744-40f9-893d-9d3880a168a4" (UID: "9c3ded19-a744-40f9-893d-9d3880a168a4"). InnerVolumeSpecName "kube-api-access-mdr72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 17:52:41.414742 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.414705    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdr72\" (UniqueName: \"kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-kube-api-access-mdr72\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:52:41.414742 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.414732    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3ded19-a744-40f9-893d-9d3880a168a4-istiod-ca-cert\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:52:41.414742 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.414742    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-envoy\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:52:41.414742 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.414751    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-token\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:52:41.415081 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.414759    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-podinfo\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:52:41.415081 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.414766    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-istio-data\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:52:41.415081 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.414774    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3ded19-a744-40f9-893d-9d3880a168a4-workload-socket\") on node \"ip-10-0-137-134.ec2.internal\" DevicePath \"\""
Jul 14 17:52:41.626555 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.626461    2580 generic.go:356] "Generic (PLEG): container finished" podID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerID="65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7" exitCode=0
Jul 14 17:52:41.626555 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.626508    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"
Jul 14 17:52:41.626555 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.626538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" event={"ID":"9c3ded19-a744-40f9-893d-9d3880a168a4","Type":"ContainerDied","Data":"65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7"}
Jul 14 17:52:41.626791 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.626590    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh" event={"ID":"9c3ded19-a744-40f9-893d-9d3880a168a4","Type":"ContainerDied","Data":"6516e9c4da6b04a9f988cf2e772a1adf7e0199521908e1c513a08534f8c3546a"}
Jul 14 17:52:41.626791 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.626612    2580 scope.go:117] "RemoveContainer" containerID="65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7"
Jul 14 17:52:41.636736 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.636712    2580 scope.go:117] "RemoveContainer" containerID="5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846"
Jul 14 17:52:41.645365 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.645343    2580 scope.go:117] "RemoveContainer" containerID="65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7"
Jul 14 17:52:41.645632 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:52:41.645611    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7\": container with ID starting with 65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7 not found: ID does not exist" containerID="65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7"
Jul 14 17:52:41.645688 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.645646    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7"} err="failed to get container status \"65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7\": rpc error: code = NotFound desc = could not find container \"65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7\": container with ID starting with 65b4481997942a9a2205c096f34dcbd53764d911b09e0346ac812c11f3d33af7 not found: ID does not exist"
Jul 14 17:52:41.645688 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.645672    2580 scope.go:117] "RemoveContainer" containerID="5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846"
Jul 14 17:52:41.645951 ip-10-0-137-134 kubenswrapper[2580]: E0714 17:52:41.645932    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846\": container with ID starting with 5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846 not found: ID does not exist" containerID="5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846"
Jul 14 17:52:41.646012 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.645962    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846"} err="failed to get container status \"5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846\": rpc error: code = NotFound desc = could not find container \"5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846\": container with ID starting with 5af8475ae9ec63e071955e53621d9063043aae6cd4c3686360bf04b736ae8846 not found: ID does not exist"
Jul 14 17:52:41.650781 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.650758    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"]
Jul 14 17:52:41.656296 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.656277    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-df3dc5-openshift-default-85bfd79f9f-ph7nh"]
Jul 14 17:52:41.717710 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.717676    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-sv6wr_efacff4e-e30c-41bf-ad77-d5b30d0323d8/manager/0.log"
Jul 14 17:52:41.853009 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.852976    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gh5jg_6706a086-5283-4c17-815b-f8a38703ab5d/manager/0.log"
Jul 14 17:52:41.882173 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:41.882088    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" path="/var/lib/kubelet/pods/9c3ded19-a744-40f9-893d-9d3880a168a4/volumes"
Jul 14 17:52:42.101474 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:42.101438    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-n5z6s_21143ebd-b32e-4e43-9584-1fec4ff6e44d/server/0.log"
Jul 14 17:52:42.243978 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:42.243939    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cg4vk_6507a330-6394-495b-b252-d73fc30b83f1/manager/0.log"
Jul 14 17:52:42.357956 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:42.357926    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-k7fk7_88fee8f0-8236-4d6a-95aa-1355865b465a/copy-manifests/0.log"
Jul 14 17:52:42.368917 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:42.368886    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-k7fk7_88fee8f0-8236-4d6a-95aa-1355865b465a/manager/0.log"
Jul 14 17:52:43.010919 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:43.010889    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54675d56c-659bs_ed7e0b20-692f-43a1-9e55-4aa569e4a031/maas-api/0.log"
Jul 14 17:52:43.132751 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:43.132707    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734155-swwsx_13eb8d8d-88ac-4194-baf1-4b439e86d649/cleanup/0.log"
Jul 14 17:52:46.331808 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:46.331774    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rmww5_86c4237a-cc6e-4bbf-8806-3f5ad859a54f/manager/0.log"
Jul 14 17:52:46.447716 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:46.447687    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-sqltf_18bee717-e3de-45f7-a7e2-da8e14181a46/manager/0.log"
Jul 14 17:52:46.559262 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:46.559229    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-fptzb_b1ce680d-2369-4f20-b262-4b54934a0621/kuadrant-console-plugin/0.log"
Jul 14 17:52:46.682935 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:46.682836    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-clqr2_5f5c9fa8-58fb-48c9-b3c8-0fa3a0d486cf/registry-server/0.log"
Jul 14 17:52:46.921956 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:46.921920    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-77fzw_8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f/limitador/0.log"
Jul 14 17:52:47.055322 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:47.055286    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4vtpt_4768c4fc-f8dd-4f8c-84cc-c2e4260706e5/manager/0.log"
Jul 14 17:52:47.661249 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:47.661212    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c548b7c75-4pfqj_1e40ced1-e25e-4f30-a89e-5bc71341474c/kube-auth-proxy/0.log"
Jul 14 17:52:47.904171 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:47.904119    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nd7hw_80fadef6-b849-463d-9ead-35c0ca1bc8cd/istio-proxy/1.log"
Jul 14 17:52:48.037510 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:48.037200    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-zsl6p_7560cab2-d533-4a56-8a9d-497347e810de/payload-pre-processing/0.log"
Jul 14 17:52:48.162422 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:48.162379    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-z6rq6_baac413f-f096-4cf1-98ba-bd5b8a8dc8f7/payload-processing/0.log"
Jul 14 17:52:48.624109 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:48.624064    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-2sjqk_c9efe483-82ff-4ef4-b89a-dec4e3f1ffca/main/0.log"
Jul 14 17:52:49.083521 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:49.083480    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-f4wvb_ff829063-494f-44ff-8d88-bc8a7e0270dd/main/0.log"
Jul 14 17:52:56.339536 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:56.339504    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t8zcm_ccda64ad-05c5-4aff-95f2-ea99d638c5aa/global-pull-secret-syncer/0.log"
Jul 14 17:52:56.487139 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:56.487107    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x28gh_46d1e4d2-c328-4d86-a524-0b64bbc9fdf0/konnectivity-agent/0.log"
Jul 14 17:52:56.573133 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:52:56.573104    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-134.ec2.internal_319818a0820800557405ab012a22c360/haproxy/0.log"
Jul 14 17:53:02.106605 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:02.106567    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rmww5_86c4237a-cc6e-4bbf-8806-3f5ad859a54f/manager/0.log"
Jul 14 17:53:02.134809 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:02.134778    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-sqltf_18bee717-e3de-45f7-a7e2-da8e14181a46/manager/0.log"
Jul 14 17:53:02.172738 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:02.172714    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-fptzb_b1ce680d-2369-4f20-b262-4b54934a0621/kuadrant-console-plugin/0.log"
Jul 14 17:53:02.237918 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:02.237884    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-clqr2_5f5c9fa8-58fb-48c9-b3c8-0fa3a0d486cf/registry-server/0.log"
Jul 14 17:53:02.452104 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:02.452076    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-77fzw_8f4af9a4-0d4c-4fd4-bf99-23c7a5fa887f/limitador/0.log"
Jul 14 17:53:02.559101 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:02.559071    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4vtpt_4768c4fc-f8dd-4f8c-84cc-c2e4260706e5/manager/0.log"
Jul 14 17:53:04.593077 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:04.593038    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-58d965cb7-hzn25_744a6ce0-c029-4ff7-a3e3-81e27b11d7c5/metrics-server/0.log"
Jul 14 17:53:04.746313 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:04.746280    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m66gz_1a4a7edb-f4ed-4b31-9e02-e0e4fc116801/node-exporter/0.log"
Jul 14 17:53:04.770434 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:04.770406    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m66gz_1a4a7edb-f4ed-4b31-9e02-e0e4fc116801/kube-rbac-proxy/0.log"
Jul 14 17:53:04.794141 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:04.794118    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m66gz_1a4a7edb-f4ed-4b31-9e02-e0e4fc116801/init-textfile/0.log"
Jul 14 17:53:05.192639 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:05.192604    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-2cqrq_6ca7ace3-a7f3-4381-a322-ce22907f0bd7/prometheus-operator/0.log"
Jul 14 17:53:05.216465 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:05.216440    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-2cqrq_6ca7ace3-a7f3-4381-a322-ce22907f0bd7/kube-rbac-proxy/0.log"
Jul 14 17:53:06.072684 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.072645    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"]
Jul 14 17:53:06.073142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073042    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbf750c-38c1-4ebb-9b4e-b2e841287bb3" containerName="test-curl-7335-140017961548464"
Jul 14 17:53:06.073142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073054    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbf750c-38c1-4ebb-9b4e-b2e841287bb3" containerName="test-curl-7335-140017961548464"
Jul 14 17:53:06.073142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073065    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy"
Jul 14 17:53:06.073142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073071    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy"
Jul 14 17:53:06.073142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073081    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54fca8c-858a-42ad-b778-d0b2525e2f5a" containerName="main"
Jul 14 17:53:06.073142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073086    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54fca8c-858a-42ad-b778-d0b2525e2f5a" containerName="main"
Jul 14 17:53:06.073142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073093    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49a9a556-24a9-46fd-afb9-38b324380a16" containerName="maas-api"
Jul 14 17:53:06.073142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073098    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="49a9a556-24a9-46fd-afb9-38b324380a16" containerName="maas-api"
Jul 14 17:53:06.073142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073104    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy"
Jul 14 17:53:06.073142 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073110    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy"
Jul 14 17:53:06.073516 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073163    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54fca8c-858a-42ad-b778-d0b2525e2f5a" containerName="main"
Jul 14 17:53:06.073516 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073174    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy"
Jul 14 17:53:06.073516 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073181    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="49a9a556-24a9-46fd-afb9-38b324380a16" containerName="maas-api"
Jul 14 17:53:06.073516 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073187    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c3ded19-a744-40f9-893d-9d3880a168a4" containerName="istio-proxy"
Jul 14 17:53:06.073516 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.073194    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dbf750c-38c1-4ebb-9b4e-b2e841287bb3" containerName="test-curl-7335-140017961548464"
Jul 14 17:53:06.076079 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.076062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.079562 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.079540    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bv25z\"/\"default-dockercfg-lld7c\""
Jul 14 17:53:06.079958 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.079941    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bv25z\"/\"openshift-service-ca.crt\""
Jul 14 17:53:06.080497 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.080484    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bv25z\"/\"kube-root-ca.crt\""
Jul 14 17:53:06.086154 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.086129    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"]
Jul 14 17:53:06.141396 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.141352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-proc\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.141396 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.141398    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp2d4\" (UniqueName: \"kubernetes.io/projected/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-kube-api-access-fp2d4\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.141638 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.141465    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-lib-modules\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.141638 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.141529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-sys\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.141638 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.141586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-podres\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.242153 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.242116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-lib-modules\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.242330 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.242179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-sys\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.242330 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.242207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-podres\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.242330 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.242256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-proc\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.242330 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.242310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-proc\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.242469 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.242322    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-lib-modules\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.242469 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.242322    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-sys\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.242469 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.242350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp2d4\" (UniqueName: \"kubernetes.io/projected/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-kube-api-access-fp2d4\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.242469 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.242383    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-podres\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.255151 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.255120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp2d4\" (UniqueName: \"kubernetes.io/projected/5fdcde4d-2f4d-486f-b8fd-5db29113db6d-kube-api-access-fp2d4\") pod \"perf-node-gather-daemonset-gxhgn\" (UID: \"5fdcde4d-2f4d-486f-b8fd-5db29113db6d\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.386800 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.386716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.527009 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.526980    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"]
Jul 14 17:53:06.529180 ip-10-0-137-134 kubenswrapper[2580]: W0714 17:53:06.529149    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5fdcde4d_2f4d_486f_b8fd_5db29113db6d.slice/crio-dc2e31945a5c9360ab08bb16bd017185369732a5f9ac36fdab2488f3cbeb5d1b WatchSource:0}: Error finding container dc2e31945a5c9360ab08bb16bd017185369732a5f9ac36fdab2488f3cbeb5d1b: Status 404 returned error can't find the container with id dc2e31945a5c9360ab08bb16bd017185369732a5f9ac36fdab2488f3cbeb5d1b
Jul 14 17:53:06.750081 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.750042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn" event={"ID":"5fdcde4d-2f4d-486f-b8fd-5db29113db6d","Type":"ContainerStarted","Data":"d53224f8115dae8dec3b3d09609bc316aa161d4dc4745cbb7a59b54a3cec2ea7"}
Jul 14 17:53:06.750261 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.750104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn" event={"ID":"5fdcde4d-2f4d-486f-b8fd-5db29113db6d","Type":"ContainerStarted","Data":"dc2e31945a5c9360ab08bb16bd017185369732a5f9ac36fdab2488f3cbeb5d1b"}
Jul 14 17:53:06.750313 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.750262    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:06.778474 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:06.778419    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn" podStartSLOduration=0.77840394 podStartE2EDuration="778.40394ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:53:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:53:06.77727636 +0000 UTC m=+2873.524876255" watchObservedRunningTime="2026-07-14 17:53:06.77840394 +0000 UTC m=+2873.526003816"
Jul 14 17:53:07.870069 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:07.870033    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-545bcf8cf8-hzhsz_3d22affd-6642-420f-89c6-61d3bb94b05c/console/0.log"
Jul 14 17:53:09.729889 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:09.729831    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-92pf7_fa711789-8cf8-4e24-86ca-da997986b21b/dns/0.log"
Jul 14 17:53:09.754485 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:09.754455    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-92pf7_fa711789-8cf8-4e24-86ca-da997986b21b/kube-rbac-proxy/0.log"
Jul 14 17:53:09.841540 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:09.841512    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8wp2t_f1bb8675-6bf3-4e71-b94d-8d3f0af7f88e/dns-node-resolver/0.log"
Jul 14 17:53:10.527283 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:10.527255    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ksph_4cefdd01-3531-4e69-89da-a20d63e8f853/node-ca/0.log"
Jul 14 17:53:11.796981 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:11.796950    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c548b7c75-4pfqj_1e40ced1-e25e-4f30-a89e-5bc71341474c/kube-auth-proxy/0.log"
Jul 14 17:53:11.861743 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:11.861707    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nd7hw_80fadef6-b849-463d-9ead-35c0ca1bc8cd/istio-proxy/0.log"
Jul 14 17:53:11.924294 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:11.924257    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nd7hw_80fadef6-b849-463d-9ead-35c0ca1bc8cd/istio-proxy/1.log"
Jul 14 17:53:11.966350 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:11.966320    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-zsl6p_7560cab2-d533-4a56-8a9d-497347e810de/payload-pre-processing/0.log"
Jul 14 17:53:12.008199 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:12.008166    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-z6rq6_baac413f-f096-4cf1-98ba-bd5b8a8dc8f7/payload-processing/0.log"
Jul 14 17:53:12.765944 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:12.765916    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-gxhgn"
Jul 14 17:53:12.774008 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:12.773976    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-txz5w_cd76e29d-8191-4fba-ab9f-9d2ba2ec5f94/serve-healthcheck-canary/0.log"
Jul 14 17:53:13.457862 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:13.457817    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jc8t9_4b13c40c-b85c-471d-9488-45f98f89c072/kube-rbac-proxy/0.log"
Jul 14 17:53:13.480112 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:13.480082    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jc8t9_4b13c40c-b85c-471d-9488-45f98f89c072/exporter/0.log"
Jul 14 17:53:13.505294 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:13.505261    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jc8t9_4b13c40c-b85c-471d-9488-45f98f89c072/extractor/0.log"
Jul 14 17:53:16.126601 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:16.126570    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-sv6wr_efacff4e-e30c-41bf-ad77-d5b30d0323d8/manager/0.log"
Jul 14 17:53:16.387823 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:16.387708    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gh5jg_6706a086-5283-4c17-815b-f8a38703ab5d/manager/0.log"
Jul 14 17:53:16.540541 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:16.540513    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-n5z6s_21143ebd-b32e-4e43-9584-1fec4ff6e44d/server/0.log"
Jul 14 17:53:16.662986 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:16.662891    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cg4vk_6507a330-6394-495b-b252-d73fc30b83f1/manager/0.log"
Jul 14 17:53:16.694036 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:16.694010    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-k7fk7_88fee8f0-8236-4d6a-95aa-1355865b465a/manager/0.log"
Jul 14 17:53:16.716650 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:16.716626    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-k7fk7_88fee8f0-8236-4d6a-95aa-1355865b465a/copy-manifests/0.log"
Jul 14 17:53:16.915923 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:16.915820    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c548b7c75-4pfqj_1e40ced1-e25e-4f30-a89e-5bc71341474c/kube-auth-proxy/0.log"
Jul 14 17:53:16.966350 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:16.966320    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nd7hw_80fadef6-b849-463d-9ead-35c0ca1bc8cd/istio-proxy/0.log"
Jul 14 17:53:17.025807 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:17.025777    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nd7hw_80fadef6-b849-463d-9ead-35c0ca1bc8cd/istio-proxy/1.log"
Jul 14 17:53:17.108060 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:17.108028    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-zsl6p_7560cab2-d533-4a56-8a9d-497347e810de/payload-pre-processing/0.log"
Jul 14 17:53:17.150537 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:17.150511    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-z6rq6_baac413f-f096-4cf1-98ba-bd5b8a8dc8f7/payload-processing/0.log"
Jul 14 17:53:18.697606 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:18.697574    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c548b7c75-4pfqj_1e40ced1-e25e-4f30-a89e-5bc71341474c/kube-auth-proxy/0.log"
Jul 14 17:53:18.749988 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:18.749958    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nd7hw_80fadef6-b849-463d-9ead-35c0ca1bc8cd/istio-proxy/0.log"
Jul 14 17:53:18.813179 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:18.813143    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nd7hw_80fadef6-b849-463d-9ead-35c0ca1bc8cd/istio-proxy/1.log"
Jul 14 17:53:18.853474 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:18.853450    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-zsl6p_7560cab2-d533-4a56-8a9d-497347e810de/payload-pre-processing/0.log"
Jul 14 17:53:18.894087 ip-10-0-137-134 kubenswrapper[2580]: I0714 17:53:18.894055    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-z6rq6_baac413f-f096-4cf1-98ba-bd5b8a8dc8f7/payload-processing/0.log"