Jul 12 18:23:22.745993 ip-10-0-130-164 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 12 18:23:22.746004 ip-10-0-130-164 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 12 18:23:22.746011 ip-10-0-130-164 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 12 18:23:22.746290 ip-10-0-130-164 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 12 18:23:32.374145 ip-10-0-130-164 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 12 18:23:32.374159 ip-10-0-130-164 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 17976b74a85a4721bd5645347776d974 --
Jul 12 18:25:17.006757 ip-10-0-130-164 systemd[1]: Starting Kubernetes Kubelet...
Jul 12 18:25:17.384361 ip-10-0-130-164 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 12 18:25:17.384361 ip-10-0-130-164 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 12 18:25:17.384361 ip-10-0-130-164 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 12 18:25:17.384361 ip-10-0-130-164 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 12 18:25:17.384361 ip-10-0-130-164 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 12 18:25:17.385197 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.385115    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 12 18:25:17.390267 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390252    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 18:25:17.390267 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390267    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390272    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390275    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390280    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390283    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390286    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390289    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390292    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390297    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390301    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390304    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390307    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390310    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390313    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390315    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390318    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390321    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390324    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390327    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 18:25:17.390332 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390330    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390332    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390335    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390338    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390342    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390346    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390349    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390351    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390354    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390357    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390359    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390362    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390365    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390367    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390370    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390373    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390375    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390378    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390381    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390386    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 18:25:17.390825 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390389    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390393    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390395    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390398    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390401    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390404    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390407    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390410    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390412    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390415    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390418    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390421    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390424    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390427    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390431    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390434    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390437    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390440    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390443    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 18:25:17.391344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390447    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390450    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390453    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390456    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390459    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390462    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390465    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390468    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390471    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390474    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390477    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390480    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390485    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390488    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390491    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390494    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390497    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390500    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390502    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390505    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 18:25:17.391857 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390508    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390511    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390513    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390516    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390518    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390521    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390524    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390542    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390546    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390549    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390552    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390556    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390559    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390562    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390565    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390568    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390570    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390573    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390576    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.390579    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 18:25:17.392363 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391371    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391379    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391383    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391386    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391389    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391394    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391399    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391402    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391405    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391409    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391411    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391414    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391417    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391420    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391423    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391426    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391429    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391432    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391435    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391439    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 18:25:17.392919 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391442    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391445    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391448    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391450    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391453    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391456    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391459    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391462    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391464    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391467    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391470    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391473    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391476    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391479    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391481    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391484    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391487    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391489    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391493    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391496    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 18:25:17.393441 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391499    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391501    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391504    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391507    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391510    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391513    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391515    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391519    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391523    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391526    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391528    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391548    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391551    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391554    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391557    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391560    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391563    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391565    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391568    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391571    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 18:25:17.393966 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391574    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391577    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391580    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391583    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391586    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391588    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391592    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391595    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391598    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391601    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391603    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391607    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391611    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391615    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391619    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391622    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391625    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391628    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391633    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 18:25:17.394481 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391637    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391640    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391652    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391656    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391659    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391662    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391665    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391668    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391670    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391673    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391676    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391679    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391682    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391684    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391687    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391690    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391693    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391696    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391698    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.391701    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 18:25:17.395006 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392102    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392107    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392110    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392113    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392117    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392120    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392123    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392126    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392129    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392132    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392136    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392138    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392141    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392144    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392147    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392150    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392154    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392156    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392159    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 18:25:17.395526 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392162    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392165    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392168    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392171    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392174    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392176    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392181    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392185    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392188    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392191    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392194    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392197    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392199    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392202    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392205    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392207    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392210    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392213    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392217    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392221    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 18:25:17.396037 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392224    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392227    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392230    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392233    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392236    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392239    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392243    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392246    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392249    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392252    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392256    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392258    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392261    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392264    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392267    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392270    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392272    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392275    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392279    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 18:25:17.396606 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392282    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392284    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392287    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392290    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392292    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392295    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392298    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392301    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392303    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392306    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392308    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392311    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392315    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392317    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392320    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392325    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392328    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392331    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392334    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392336    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 18:25:17.397104 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392339    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392344    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392346    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392349    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392352    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392354    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392357    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392360    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392362    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392366    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392369    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392372    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392374    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392377    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392380    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392383    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392386    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392388    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392391    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392394    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 18:25:17.397641 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.392396    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392466    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392472    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392478    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392483    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392487    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392491    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392497    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392501    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392504    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392508    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392511    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392515    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392521    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392524    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392527    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392543    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392546    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392549    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392554    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392557    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392561    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392564    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392567    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 12 18:25:17.398190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392571    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392574    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392577    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392581    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392584    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392587    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392590    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392593    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392596    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392600    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392604    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392607    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392610    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392613    2562 flags.go:64] FLAG: --enable-server="true"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392617    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392622    2562 flags.go:64] FLAG: --event-burst="100"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392625    2562 flags.go:64] FLAG: --event-qps="50"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392628    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392632    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392635    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392639    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392643    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392647    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392650    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392653    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 12 18:25:17.398840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392656    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392659    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392662    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392665    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392668    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392672    2562 flags.go:64] FLAG: --feature-gates=""
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392675    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392678    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392681    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392685    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392688    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392691    2562 flags.go:64] FLAG: --help="false"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392694    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392697    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392701    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392704    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392707    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392711    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392714    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392717    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392720    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392723    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392727    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392730    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 12 18:25:17.399474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392734    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392737    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392740    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392743    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392751    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392754    2562 flags.go:64] FLAG: --lock-file=""
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392757    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392760    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392764    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392769    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392772    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392775    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392779    2562 flags.go:64] FLAG: --logging-format="text"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392790    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392793    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392796    2562 flags.go:64] FLAG: --manifest-url=""
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392800    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392804    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392808    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392811    2562 flags.go:64] FLAG: --max-pods="110"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392814    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392817    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392821    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392824    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392827    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 12 18:25:17.400103 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392830    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392833    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392841    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392844    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392847    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392850    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392854    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392859    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392863    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392866    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392869    2562 flags.go:64] FLAG: --port="10250"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392874    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392877    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-068865ef7b2c8a838"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392880    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392883    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392886    2562 flags.go:64] FLAG: --register-node="true"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392889    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392893    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392895    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392899    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392902    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392905    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392909    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392912    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392915    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 12 18:25:17.400794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392918    2562 flags.go:64] FLAG: --runonce="false"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392921    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392924    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392927    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392931    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392934    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392937    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392940    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392943    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392946    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392949    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392952    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392955    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392960    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392964    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392967    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392972    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392975    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392979    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392983    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392986    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392989    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392992    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392995    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.392998    2562 flags.go:64] FLAG: --v="2"
Jul 12 18:25:17.401433 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.393002    2562 flags.go:64] FLAG: --version="false"
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.393006    2562 flags.go:64] FLAG: --vmodule=""
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.393010    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.393013    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393159    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393164    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393167    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393170    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393173    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393177    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393182    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393185    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393188    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393191    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393193    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393196    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393199    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393201    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393204    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393207    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 18:25:17.402130 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393211    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393214    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393218    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393220    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393223    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393228    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393231    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393234    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393237    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393239    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393243    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393246    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393248    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393251    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393254    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393256    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393259    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393262    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393265    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393267    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 18:25:17.402701 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393270    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393273    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393276    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393280    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393283    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393286    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393289    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393292    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393295    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393298    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393301    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393304    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393309    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393312    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393315    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393318    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393321    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393325    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393329    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 18:25:17.403214 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393332    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393334    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393337    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393340    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393343    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393346    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393348    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393351    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393354    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393357    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393359    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393362    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393365    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393367    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393370    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393373    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393375    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393378    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393380    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 18:25:17.403729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393383    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393385    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393388    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393391    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393393    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393396    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393401    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393404    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393406    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393409    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393412    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393417    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393420    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393423    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393425    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393428    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393432    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393435    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393438    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393441    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 18:25:17.404216 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393444    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393447    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393450    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393453    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.393456    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.394051    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.402770    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.402789    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.402803    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.402817    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402886    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402892    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402896    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402899    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402902    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 18:25:17.404755 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402905    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402908    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402911    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402914    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402917    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402919    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402922    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402926    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402928    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402931    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402933    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402936    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402946    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402949    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402952    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402955    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402958    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402961    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402965    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 18:25:17.405155 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402970    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402973    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402976    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402980    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402982    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402993    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.402997    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403000    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403004    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403007    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403010    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403013    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403016    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403018    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403021    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403024    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403026    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403029    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403032    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403035    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 18:25:17.405674 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403037    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403040    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403043    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403045    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403048    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403050    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403054    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403057    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403060    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403062    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403065    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403068    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403071    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403075    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403078    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403080    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403083    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403086    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403096    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403099    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 18:25:17.406185 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403102    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403104    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403107    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403110    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403112    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403115    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403118    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403120    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403123    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403126    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403129    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403131    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403134    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403136    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403139    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403141    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403144    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403147    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403149    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 18:25:17.406716 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403152    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403155    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403158    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403160    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403163    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403167    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403171    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403175    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403178    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403181    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403183    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403186    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403195    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403198    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403200    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 18:25:17.407204 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403203    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.403208    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403370    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403375    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403378    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403381    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403384    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403387    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403391    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403396    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403399    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403402    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403405    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403408    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 18:25:17.407867 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403411    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403413    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403416    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403419    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403422    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403425    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403428    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403431    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403434    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403437    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403440    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403442    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403445    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403448    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403451    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403455    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403464    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403467    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403470    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403473    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 18:25:17.408389 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403476    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403479    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403481    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403484    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403486    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403489    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403492    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403494    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403499    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403502    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403506    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403509    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403512    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403515    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403518    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403522    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403525    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403543    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403549    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403572    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 18:25:17.409171 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403589    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403598    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403601    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403604    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403607    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403609    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403612    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403615    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403618    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403629    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403632    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403635    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403637    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403640    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403643    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403646    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403648    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403651    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403654    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403657    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 18:25:17.409744 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403660    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403663    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403666    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403668    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403671    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403674    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403677    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403679    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403682    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403685    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403688    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403690    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403693    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403696    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403699    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403702    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403705    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403708    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403710    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403713    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 18:25:17.410269 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403716    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403718    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403727    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403730    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403733    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403736    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:17.403739    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.403744    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.403876    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.408925    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.410692    2562 server.go:1019] "Starting client certificate rotation"
Jul 12 18:25:17.410815 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.410803    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 12 18:25:17.411127 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.410842    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 12 18:25:17.413446 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.413432    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 12 18:25:17.415916 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.415900    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 12 18:25:17.428319 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.428302    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 12 18:25:17.430391 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.430374    2562 log.go:25] "Validated CRI v1 image API"
Jul 12 18:25:17.431311 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.431295    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 12 18:25:17.436244 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.436209    2562 fs.go:135] Filesystem UUIDs: map[60375d46-068e-435d-be94-8a6e555e0c34:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b35cfb4f-c177-4178-bfd2-aa36a831f8a2:/dev/nvme0n1p3]
Jul 12 18:25:17.436313 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.436246    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 12 18:25:17.441817 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.441698    2562 manager.go:216] Machine: {Timestamp:2026-07-12 18:25:17.440523382 +0000 UTC m=+0.323873649 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096866 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2276d6fa928f6e18ee42bf7e368b17 SystemUUID:ec2276d6-fa92-8f6e-18ee-42bf7e368b17 BootID:17976b74-a85a-4721-bd56-45347776d974 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:63:53:be:12:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:63:53:be:12:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:bd:f9:87:42:6a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 12 18:25:17.441817 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.441812    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 12 18:25:17.441941 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.441901    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 12 18:25:17.445207 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.445177    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 12 18:25:17.445386 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.445214    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-164.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 12 18:25:17.446019 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.446005    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 12 18:25:17.446087 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.446024    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 12 18:25:17.446087 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.446034    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 12 18:25:17.446611 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.446589    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 12 18:25:17.446735 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.446720    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 12 18:25:17.448090 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.448077    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 12 18:25:17.451194 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.451178    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 12 18:25:17.451349 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.451337    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 12 18:25:17.451415 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.451376    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 12 18:25:17.451415 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.451391    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 12 18:25:17.451415 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.451408    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 12 18:25:17.451558 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.451423    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 12 18:25:17.451558 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.451436    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 12 18:25:17.451943 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.451924    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 12 18:25:17.452037 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.451956    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 12 18:25:17.455694 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.455592    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 12 18:25:17.457691 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.457675    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 12 18:25:17.457758 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.457708    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 12 18:25:17.457883 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.457872    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 12 18:25:17.457926 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.457888    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 12 18:25:17.457926 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.457895    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 12 18:25:17.457926 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.457901    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 12 18:25:17.457926 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.457910    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 12 18:25:17.457926 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.457916    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 12 18:25:17.458549 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.458525    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 12 18:25:17.458581 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.458551    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 12 18:25:17.458581 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.458559    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 12 18:25:17.458581 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.458565    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 12 18:25:17.458581 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.458574    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 12 18:25:17.458581 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.458582    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 12 18:25:17.459568 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.459559    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 12 18:25:17.459606 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.459569    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 12 18:25:17.463179 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.463161    2562 server.go:1268] "Started kubelet"
Jul 12 18:25:17.463281 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.463228    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 12 18:25:17.463528 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.463475    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 12 18:25:17.463652 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.463572    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 12 18:25:17.464325 ip-10-0-130-164 systemd[1]: Started Kubernetes Kubelet.
Jul 12 18:25:17.465153 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.465129    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 12 18:25:17.465244 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.465157    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 12 18:25:17.465244 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.465188    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 12 18:25:17.465732 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.465708    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 12 18:25:17.467985 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.467969    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 12 18:25:17.468203 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.467235    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-164.ec2.internal.18c19e0fa60f0a93  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-164.ec2.internal,UID:ip-10-0-130-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-164.ec2.internal,},FirstTimestamp:2026-07-12 18:25:17.463128723 +0000 UTC m=+0.346478997,LastTimestamp:2026-07-12 18:25:17.463128723 +0000 UTC m=+0.346478997,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-164.ec2.internal,}"
Jul 12 18:25:17.469443 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.469423    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 12 18:25:17.470286 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.470270    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hpbfp"
Jul 12 18:25:17.473215 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.473194    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 12 18:25:17.473329 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.473214    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 12 18:25:17.473329 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.473321    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:17.474040 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474021    2562 factory.go:55] Registering systemd factory
Jul 12 18:25:17.474040 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474037    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 12 18:25:17.474161 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474047    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 12 18:25:17.474161 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474065    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 12 18:25:17.474161 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474147    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 12 18:25:17.474161 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474150    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 12 18:25:17.474161 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474156    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 12 18:25:17.474459 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474278    2562 factory.go:153] Registering CRI-O factory
Jul 12 18:25:17.474459 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474294    2562 factory.go:223] Registration of the crio container factory successfully
Jul 12 18:25:17.474459 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474351    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 12 18:25:17.474459 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474374    2562 factory.go:103] Registering Raw factory
Jul 12 18:25:17.474459 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.474390    2562 manager.go:1196] Started watching for new ooms in manager
Jul 12 18:25:17.475032 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.475014    2562 manager.go:349] Starting recovery of all containers
Jul 12 18:25:17.476365 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.476345    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hpbfp"
Jul 12 18:25:17.484921 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.484896    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 18:25:17.488668 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.488447    2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-164.ec2.internal\" not found" node="ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.488668 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.488496    2562 manager.go:354] Recovery completed
Jul 12 18:25:17.492938 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.492924    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 18:25:17.495297 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.495279    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 18:25:17.495358 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.495313    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 18:25:17.495358 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.495324    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientPID"
Jul 12 18:25:17.495842 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.495827    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 12 18:25:17.495902 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.495844    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 12 18:25:17.495902 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.495864    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 12 18:25:17.495902 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.495830    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 18:25:17.496894 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.496878    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 18:25:17.496974 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.496908    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 18:25:17.496974 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.496922    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientPID"
Jul 12 18:25:17.497144 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.497131    2562 policy_none.go:49] "None policy: Start"
Jul 12 18:25:17.497180 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.497150    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 12 18:25:17.497180 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.497167    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 12 18:25:17.498457 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.498445    2562 policy_none.go:47] "Start"
Jul 12 18:25:17.534111 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.534093    2562 manager.go:337] "Starting Device Plugin manager"
Jul 12 18:25:17.534176 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.534139    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 12 18:25:17.534176 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.534150    2562 server.go:87] "Starting device plugin registration server"
Jul 12 18:25:17.534406 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.534395    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 12 18:25:17.534455 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.534407    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 12 18:25:17.534608 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.534585    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 12 18:25:17.534715 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.534685    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 12 18:25:17.534715 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.534696    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 12 18:25:17.535422 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.535395    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 12 18:25:17.535514 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.535505    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:17.594879 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.594840    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 12 18:25:17.598189 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.596192    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 12 18:25:17.598189 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.596238    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 12 18:25:17.598189 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.596264    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 12 18:25:17.598189 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.596357    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 12 18:25:17.599061 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.599039    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 18:25:17.634629 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.634607    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 18:25:17.635523 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.635506    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 18:25:17.635602 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.635536    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 18:25:17.635602 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.635548    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientPID"
Jul 12 18:25:17.635602 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.635583    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.644240 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.644222    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.644309 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.644246    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-164.ec2.internal\": node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:17.661722 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.661699    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:17.697990 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.697942    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal"]
Jul 12 18:25:17.698091 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.698032    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 18:25:17.699120 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.699102    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 18:25:17.699211 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.699140    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 18:25:17.699211 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.699161    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientPID"
Jul 12 18:25:17.700656 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.700633    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 18:25:17.700810 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.700795    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.700861 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.700843    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 18:25:17.701510 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.701492    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 18:25:17.701623 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.701519    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 18:25:17.701623 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.701540    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 18:25:17.701623 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.701552    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientPID"
Jul 12 18:25:17.701623 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.701569    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 18:25:17.701623 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.701585    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientPID"
Jul 12 18:25:17.702841 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.702826    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.702899 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.702856    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 18:25:17.704035 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.704017    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 18:25:17.704118 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.704045    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 18:25:17.704118 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.704060    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeHasSufficientPID"
Jul 12 18:25:17.726206 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.726189    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-164.ec2.internal\" not found" node="ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.730985 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.730969    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-164.ec2.internal\" not found" node="ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.762070 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.762040    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:17.862353 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.862264    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:17.876101 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.876075    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d30db9f1d57f95317b299b4a177ebe3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal\" (UID: \"1d30db9f1d57f95317b299b4a177ebe3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.876101 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.876105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d30db9f1d57f95317b299b4a177ebe3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal\" (UID: \"1d30db9f1d57f95317b299b4a177ebe3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.876288 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.876128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2a48c8d39a9521700ee4336712fb4822-config\") pod \"kube-apiserver-proxy-ip-10-0-130-164.ec2.internal\" (UID: \"2a48c8d39a9521700ee4336712fb4822\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.963445 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:17.963412    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:17.976749 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.976720    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2a48c8d39a9521700ee4336712fb4822-config\") pod \"kube-apiserver-proxy-ip-10-0-130-164.ec2.internal\" (UID: \"2a48c8d39a9521700ee4336712fb4822\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.976822 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.976664    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2a48c8d39a9521700ee4336712fb4822-config\") pod \"kube-apiserver-proxy-ip-10-0-130-164.ec2.internal\" (UID: \"2a48c8d39a9521700ee4336712fb4822\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.976822 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.976791    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d30db9f1d57f95317b299b4a177ebe3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal\" (UID: \"1d30db9f1d57f95317b299b4a177ebe3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.976822 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.976813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d30db9f1d57f95317b299b4a177ebe3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal\" (UID: \"1d30db9f1d57f95317b299b4a177ebe3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.976914 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.976899    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d30db9f1d57f95317b299b4a177ebe3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal\" (UID: \"1d30db9f1d57f95317b299b4a177ebe3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:17.976952 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:17.976899    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d30db9f1d57f95317b299b4a177ebe3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal\" (UID: \"1d30db9f1d57f95317b299b4a177ebe3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:18.031591 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.031540    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:18.033802 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.033782    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:18.064455 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:18.064426    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:18.170064 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:18.169958    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:18.277672 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:18.277630    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:18.378593 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:18.378555    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-164.ec2.internal\" not found"
Jul 12 18:25:18.409494 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.409453    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 12 18:25:18.410078 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.409626    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 12 18:25:18.410078 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.409663    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 12 18:25:18.459913 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.459854    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 18:25:18.474352 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.474319    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 12 18:25:18.474494 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.474332    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:18.480347 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.480313    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 18:20:17 +0000 UTC" deadline="2028-01-20 22:07:02.156921131 +0000 UTC"
Jul 12 18:25:18.480581 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.480348    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13371h41m43.676579984s"
Jul 12 18:25:18.484607 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.484589    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 12 18:25:18.486601 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.486582    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal"
Jul 12 18:25:18.488616 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.487548    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 12 18:25:18.497010 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.496992    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 12 18:25:18.505914 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.505893    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wbfxt"
Jul 12 18:25:18.511688 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.511671    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wbfxt"
Jul 12 18:25:18.582499 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:18.582449    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a48c8d39a9521700ee4336712fb4822.slice/crio-49e230ef97a063e872882531b9d3399d461aa8c29f69406c82bdb326684b1ff9 WatchSource:0}: Error finding container 49e230ef97a063e872882531b9d3399d461aa8c29f69406c82bdb326684b1ff9: Status 404 returned error can't find the container with id 49e230ef97a063e872882531b9d3399d461aa8c29f69406c82bdb326684b1ff9
Jul 12 18:25:18.582805 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:18.582780    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d30db9f1d57f95317b299b4a177ebe3.slice/crio-24fb6b548a92ba0b6c25883a7ac533f97dd82617e28995f23aafe4927605567d WatchSource:0}: Error finding container 24fb6b548a92ba0b6c25883a7ac533f97dd82617e28995f23aafe4927605567d: Status 404 returned error can't find the container with id 24fb6b548a92ba0b6c25883a7ac533f97dd82617e28995f23aafe4927605567d
Jul 12 18:25:18.586956 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.586940    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 18:25:18.601011 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.600961    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal" event={"ID":"1d30db9f1d57f95317b299b4a177ebe3","Type":"ContainerStarted","Data":"24fb6b548a92ba0b6c25883a7ac533f97dd82617e28995f23aafe4927605567d"}
Jul 12 18:25:18.602227 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.602202    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal" event={"ID":"2a48c8d39a9521700ee4336712fb4822","Type":"ContainerStarted","Data":"49e230ef97a063e872882531b9d3399d461aa8c29f69406c82bdb326684b1ff9"}
Jul 12 18:25:18.885764 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:18.885677    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 18:25:19.452825 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.452794    2562 apiserver.go:52] "Watching apiserver"
Jul 12 18:25:19.460183 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.460159    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 12 18:25:19.460675 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.460640    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-p8p68","kube-system/global-pull-secret-syncer-4kw4m","kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr","openshift-image-registry/node-ca-rxs9c","openshift-multus/multus-additional-cni-plugins-gx47v","openshift-multus/network-metrics-daemon-cbzdr","openshift-network-diagnostics/network-check-target-q77wh","openshift-network-operator/iptables-alerter-vqgmh","kube-system/konnectivity-agent-b988b","openshift-cluster-node-tuning-operator/tuned-j5hzk","openshift-dns/node-resolver-f4xh5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal","openshift-multus/multus-q9sm4"]
Jul 12 18:25:19.462193 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.462170    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:19.462299 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.462267    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:19.463356 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.463325    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.465866 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.465845    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2dxcb\""
Jul 12 18:25:19.466122 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.466106    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 12 18:25:19.466217 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.466205    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 12 18:25:19.466349 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.466331    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.467249 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.467230    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.468878 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.468442    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.468878 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.468472    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 12 18:25:19.468878 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.468744    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 12 18:25:19.468878 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.468754    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 12 18:25:19.469176 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.468964    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l24gz\""
Jul 12 18:25:19.469176 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.469084    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 12 18:25:19.469176 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.469133    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 12 18:25:19.469609 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.469538    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 12 18:25:19.469948 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.469767    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gxlzj\""
Jul 12 18:25:19.469948 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.469767    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 12 18:25:19.470132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.470063    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.470846 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.470826    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 12 18:25:19.471302 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.471281    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 12 18:25:19.471420 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.471396    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8q5cp\""
Jul 12 18:25:19.471581 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.471491    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 12 18:25:19.472261 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.472241    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 12 18:25:19.472365 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.472295    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.472719 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.472697    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 12 18:25:19.472812 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.472697    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nlglj\""
Jul 12 18:25:19.473388 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.473333    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:19.473494 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.473401    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:19.475010 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.474984    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 12 18:25:19.476032 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.475325    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 12 18:25:19.476032 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.475518    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mtkd7\""
Jul 12 18:25:19.476931 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.476247    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 12 18:25:19.476931 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.476598    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:19.476931 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.476668    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:19.478354 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.478310    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:19.480803 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.480786    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.481670 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.481156    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 12 18:25:19.482216 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.481210    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jzw5f\""
Jul 12 18:25:19.482216 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.481882    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.482216 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.481267    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 12 18:25:19.485443 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.484091    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 12 18:25:19.485443 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.484429    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 12 18:25:19.485443 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.484859    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 12 18:25:19.485443 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485084    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 12 18:25:19.485443 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485256    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 12 18:25:19.485806 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485596    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 12 18:25:19.485806 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485607    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 12 18:25:19.485806 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dec7786a-3b72-4a67-9aea-8d1030b7cbab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.485806 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgdsg\" (UniqueName: \"kubernetes.io/projected/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-kube-api-access-qgdsg\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.485806 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f73fd2b-9f51-44df-98cb-672c606fc32e-host\") pod \"node-ca-rxs9c\" (UID: \"6f73fd2b-9f51-44df-98cb-672c606fc32e\") " pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.485806 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485695    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmscs\" (UniqueName: \"kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs\") pod \"network-check-target-q77wh\" (UID: \"d11c8121-7766-4973-9398-0ed4c73fd2e3\") " pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:19.485806 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485721    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpldb\" (UniqueName: \"kubernetes.io/projected/6f73fd2b-9f51-44df-98cb-672c606fc32e-kube-api-access-kpldb\") pod \"node-ca-rxs9c\" (UID: \"6f73fd2b-9f51-44df-98cb-672c606fc32e\") " pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.485806 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-os-release\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.485806 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485769    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-etc-kubernetes\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.485806 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485793    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-host\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485817    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-run-netns\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485845    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-registration-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485902    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99149170-b408-430b-899c-f77f30a7f163-host-slash\") pod \"iptables-alerter-vqgmh\" (UID: \"99149170-b408-430b-899c-f77f30a7f163\") " pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485934    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ff40a73-e740-4c46-9240-d5dba77aeefc-cni-binary-copy\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.485962    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-var-lib-cni-multus\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486001    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-daemon-config\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486015    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ddgp7\""
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486024    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8xdc\" (UniqueName: \"kubernetes.io/projected/6ff40a73-e740-4c46-9240-d5dba77aeefc-kube-api-access-z8xdc\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-device-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-system-cni-dir\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486131    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8dzdf\""
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486135    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486174    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dec7786a-3b72-4a67-9aea-8d1030b7cbab-cni-binary-copy\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486208    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486261    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xtgt\" (UniqueName: \"kubernetes.io/projected/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-kube-api-access-5xtgt\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6f73fd2b-9f51-44df-98cb-672c606fc32e-serviceca\") pod \"node-ca-rxs9c\" (UID: \"6f73fd2b-9f51-44df-98cb-672c606fc32e\") " pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.486320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486310    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f45dx\" (UniqueName: \"kubernetes.io/projected/dec7786a-3b72-4a67-9aea-8d1030b7cbab-kube-api-access-f45dx\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-sysctl-conf\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486354    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-systemd\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486385    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-run\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-var-lib-kubelet\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486468    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-socket-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486550    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-kubernetes\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486594    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-var-lib-kubelet\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-var-lib-cni-bin\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486647    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dec7786a-3b72-4a67-9aea-8d1030b7cbab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-modprobe-d\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-sysctl-d\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486758    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-system-cni-dir\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-conf-dir\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.487083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486805    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-cnibin\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486830    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk57z\" (UniqueName: \"kubernetes.io/projected/38c69e27-be45-4ff9-b58d-906334dd14ae-kube-api-access-bk57z\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-hostroot\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/adc887e4-8e0a-4ed3-9978-792e0a1573bb-dbus\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486901    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486930    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-lib-modules\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38c69e27-be45-4ff9-b58d-906334dd14ae-tmp\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486974    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-run-k8s-cni-cncf-io\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.486998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99149170-b408-430b-899c-f77f30a7f163-iptables-alerter-script\") pod \"iptables-alerter-vqgmh\" (UID: \"99149170-b408-430b-899c-f77f30a7f163\") " pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487022    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-sys\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487044    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-tuned\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487065    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-cni-dir\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487088    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-cnibin\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487114    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-socket-dir-parent\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487139    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-sys-fs\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvxlm\" (UniqueName: \"kubernetes.io/projected/99149170-b408-430b-899c-f77f30a7f163-kube-api-access-bvxlm\") pod \"iptables-alerter-vqgmh\" (UID: \"99149170-b408-430b-899c-f77f30a7f163\") " pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.487642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487193    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-sysconfig\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.488375 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487216    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-os-release\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.488375 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487241    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-run-multus-certs\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.488375 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.487265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/adc887e4-8e0a-4ed3-9978-792e0a1573bb-kubelet-config\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:19.513491 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.513428    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 18:20:18 +0000 UTC" deadline="2028-01-24 07:49:37.06348031 +0000 UTC"
Jul 12 18:25:19.513491 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.513459    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13453h24m17.550026037s"
Jul 12 18:25:19.522378 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.522359    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 18:25:19.574902 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.574875    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 12 18:25:19.588289 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588252    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk57z\" (UniqueName: \"kubernetes.io/projected/38c69e27-be45-4ff9-b58d-906334dd14ae-kube-api-access-bk57z\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.588438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-hostroot\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.588438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/adc887e4-8e0a-4ed3-9978-792e0a1573bb-dbus\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:19.588438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.588438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588375    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-ovn-node-metrics-cert\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.588438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588399    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-ovnkube-script-lib\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.588438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588421    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-lib-modules\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38c69e27-be45-4ff9-b58d-906334dd14ae-tmp\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588478    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-run-k8s-cni-cncf-io\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588500    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99149170-b408-430b-899c-f77f30a7f163-iptables-alerter-script\") pod \"iptables-alerter-vqgmh\" (UID: \"99149170-b408-430b-899c-f77f30a7f163\") " pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-cni-bin\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588539    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-cni-netd\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588554    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-hostroot\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-sys\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588594    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/adc887e4-8e0a-4ed3-9978-792e0a1573bb-dbus\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588599    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-tuned\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-run-k8s-cni-cncf-io\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588634    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-cni-dir\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-cnibin\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-sys\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588694    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-socket-dir-parent\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-sys-fs\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588745    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvxlm\" (UniqueName: \"kubernetes.io/projected/99149170-b408-430b-899c-f77f30a7f163-kube-api-access-bvxlm\") pod \"iptables-alerter-vqgmh\" (UID: \"99149170-b408-430b-899c-f77f30a7f163\") " pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.588786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588768    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-cnibin\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-sysconfig\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588818    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-sys-fs\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588721    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-lib-modules\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588771    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-socket-dir-parent\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588770    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-sysconfig\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-os-release\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588850    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-cni-dir\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-systemd-units\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-os-release\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588968    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-var-lib-openvswitch\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.588965    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5183a6f1-0901-4208-b85a-8fab51402e3e-agent-certs\") pod \"konnectivity-agent-b988b\" (UID: \"5183a6f1-0901-4208-b85a-8fab51402e3e\") " pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589036    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-run-multus-certs\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589082    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/adc887e4-8e0a-4ed3-9978-792e0a1573bb-kubelet-config\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589118    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-run-multus-certs\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dec7786a-3b72-4a67-9aea-8d1030b7cbab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.589630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589142    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99149170-b408-430b-899c-f77f30a7f163-iptables-alerter-script\") pod \"iptables-alerter-vqgmh\" (UID: \"99149170-b408-430b-899c-f77f30a7f163\") " pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgdsg\" (UniqueName: \"kubernetes.io/projected/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-kube-api-access-qgdsg\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589178    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/adc887e4-8e0a-4ed3-9978-792e0a1573bb-kubelet-config\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f73fd2b-9f51-44df-98cb-672c606fc32e-host\") pod \"node-ca-rxs9c\" (UID: \"6f73fd2b-9f51-44df-98cb-672c606fc32e\") " pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589229    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr999\" (UniqueName: \"kubernetes.io/projected/b098ac18-b58d-4a14-af63-7b1973beb36b-kube-api-access-zr999\") pod \"node-resolver-f4xh5\" (UID: \"b098ac18-b58d-4a14-af63-7b1973beb36b\") " pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmscs\" (UniqueName: \"kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs\") pod \"network-check-target-q77wh\" (UID: \"d11c8121-7766-4973-9398-0ed4c73fd2e3\") " pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589274    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f73fd2b-9f51-44df-98cb-672c606fc32e-host\") pod \"node-ca-rxs9c\" (UID: \"6f73fd2b-9f51-44df-98cb-672c606fc32e\") " pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpldb\" (UniqueName: \"kubernetes.io/projected/6f73fd2b-9f51-44df-98cb-672c606fc32e-kube-api-access-kpldb\") pod \"node-ca-rxs9c\" (UID: \"6f73fd2b-9f51-44df-98cb-672c606fc32e\") " pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589307    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-kubelet\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-run-openvswitch\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589356    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-env-overrides\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpmlr\" (UniqueName: \"kubernetes.io/projected/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-kube-api-access-lpmlr\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589407    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-os-release\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589431    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-etc-kubernetes\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b098ac18-b58d-4a14-af63-7b1973beb36b-tmp-dir\") pod \"node-resolver-f4xh5\" (UID: \"b098ac18-b58d-4a14-af63-7b1973beb36b\") " pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589517    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-host\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589526    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-os-release\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.590450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589547    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-run-netns\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589563    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-etc-kubernetes\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-registration-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589578    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-host\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589607    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-run-netns\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-registration-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589620    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99149170-b408-430b-899c-f77f30a7f163-host-slash\") pod \"iptables-alerter-vqgmh\" (UID: \"99149170-b408-430b-899c-f77f30a7f163\") " pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589662    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99149170-b408-430b-899c-f77f30a7f163-host-slash\") pod \"iptables-alerter-vqgmh\" (UID: \"99149170-b408-430b-899c-f77f30a7f163\") " pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589693    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dec7786a-3b72-4a67-9aea-8d1030b7cbab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589690    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-slash\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-run-ovn-kubernetes\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ff40a73-e740-4c46-9240-d5dba77aeefc-cni-binary-copy\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-var-lib-cni-multus\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-daemon-config\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589883    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8xdc\" (UniqueName: \"kubernetes.io/projected/6ff40a73-e740-4c46-9240-d5dba77aeefc-kube-api-access-z8xdc\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589897    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-var-lib-cni-multus\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-device-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.591230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-run-netns\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589959    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-run-ovn\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-system-cni-dir\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.589988    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-device-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590015    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590041    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b098ac18-b58d-4a14-af63-7b1973beb36b-hosts-file\") pod \"node-resolver-f4xh5\" (UID: \"b098ac18-b58d-4a14-af63-7b1973beb36b\") " pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590051    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-system-cni-dir\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590066    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590094    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dec7786a-3b72-4a67-9aea-8d1030b7cbab-cni-binary-copy\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-etc-openvswitch\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xtgt\" (UniqueName: \"kubernetes.io/projected/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-kube-api-access-5xtgt\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.590174    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590196    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6f73fd2b-9f51-44df-98cb-672c606fc32e-serviceca\") pod \"node-ca-rxs9c\" (UID: \"6f73fd2b-9f51-44df-98cb-672c606fc32e\") " pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590223    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f45dx\" (UniqueName: \"kubernetes.io/projected/dec7786a-3b72-4a67-9aea-8d1030b7cbab-kube-api-access-f45dx\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.590293    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret podName:adc887e4-8e0a-4ed3-9978-792e0a1573bb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:20.090229594 +0000 UTC m=+2.973579861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret") pod "global-pull-secret-syncer-4kw4m" (UID: "adc887e4-8e0a-4ed3-9978-792e0a1573bb") : object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:19.591886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-run-systemd\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590335    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ff40a73-e740-4c46-9240-d5dba77aeefc-cni-binary-copy\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5183a6f1-0901-4208-b85a-8fab51402e3e-konnectivity-ca\") pod \"konnectivity-agent-b988b\" (UID: \"5183a6f1-0901-4208-b85a-8fab51402e3e\") " pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590394    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-sysctl-conf\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590441    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-systemd\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-run\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590505    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-systemd\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590518    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-daemon-config\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-var-lib-kubelet\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590562    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-var-lib-kubelet\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-socket-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-run\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590661    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dec7786a-3b72-4a67-9aea-8d1030b7cbab-cni-binary-copy\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-kubernetes\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.592474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590715    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-var-lib-kubelet\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-var-lib-cni-bin\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.590755    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590772    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dec7786a-3b72-4a67-9aea-8d1030b7cbab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-var-lib-kubelet\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590773    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-sysctl-conf\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.590813    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs podName:77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:20.090796002 +0000 UTC m=+2.974146256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs") pod "network-metrics-daemon-cbzdr" (UID: "77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-kubernetes\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590833    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-log-socket\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590841    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590888    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-host-var-lib-cni-bin\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590950    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-modprobe-d\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-sysctl-d\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-system-cni-dir\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591051    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-conf-dir\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-modprobe-d\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591067    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6f73fd2b-9f51-44df-98cb-672c606fc32e-serviceca\") pod \"node-ca-rxs9c\" (UID: \"6f73fd2b-9f51-44df-98cb-672c606fc32e\") " pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.593115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-multus-conf-dir\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.593714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-sysctl-d\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.593714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591095    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-cnibin\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.593714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.590990    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-socket-dir\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.593714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dec7786a-3b72-4a67-9aea-8d1030b7cbab-cnibin\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.593714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591111    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ff40a73-e740-4c46-9240-d5dba77aeefc-system-cni-dir\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.593714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591164    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-node-log\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.593714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-ovnkube-config\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.593714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.591255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dec7786a-3b72-4a67-9aea-8d1030b7cbab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.593714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.592391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38c69e27-be45-4ff9-b58d-906334dd14ae-tmp\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.593714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.592472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38c69e27-be45-4ff9-b58d-906334dd14ae-etc-tuned\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.595501 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.595407    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 18:25:19.595501 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.595432    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 18:25:19.595501 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.595447    2562 projected.go:196] Error preparing data for projected volume kube-api-access-zmscs for pod openshift-network-diagnostics/network-check-target-q77wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:19.596011 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:19.595514    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs podName:d11c8121-7766-4973-9398-0ed4c73fd2e3 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:20.095499891 +0000 UTC m=+2.978850172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zmscs" (UniqueName: "kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs") pod "network-check-target-q77wh" (UID: "d11c8121-7766-4973-9398-0ed4c73fd2e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:19.597209 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.597155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk57z\" (UniqueName: \"kubernetes.io/projected/38c69e27-be45-4ff9-b58d-906334dd14ae-kube-api-access-bk57z\") pod \"tuned-j5hzk\" (UID: \"38c69e27-be45-4ff9-b58d-906334dd14ae\") " pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.597853 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.597825    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvxlm\" (UniqueName: \"kubernetes.io/projected/99149170-b408-430b-899c-f77f30a7f163-kube-api-access-bvxlm\") pod \"iptables-alerter-vqgmh\" (UID: \"99149170-b408-430b-899c-f77f30a7f163\") " pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.598520 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.598463    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgdsg\" (UniqueName: \"kubernetes.io/projected/6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2-kube-api-access-qgdsg\") pod \"aws-ebs-csi-driver-node-cgpbr\" (UID: \"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.598778 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.598755    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f45dx\" (UniqueName: \"kubernetes.io/projected/dec7786a-3b72-4a67-9aea-8d1030b7cbab-kube-api-access-f45dx\") pod \"multus-additional-cni-plugins-gx47v\" (UID: \"dec7786a-3b72-4a67-9aea-8d1030b7cbab\") " pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.599886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.599864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpldb\" (UniqueName: \"kubernetes.io/projected/6f73fd2b-9f51-44df-98cb-672c606fc32e-kube-api-access-kpldb\") pod \"node-ca-rxs9c\" (UID: \"6f73fd2b-9f51-44df-98cb-672c606fc32e\") " pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.600118 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.600095    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8xdc\" (UniqueName: \"kubernetes.io/projected/6ff40a73-e740-4c46-9240-d5dba77aeefc-kube-api-access-z8xdc\") pod \"multus-q9sm4\" (UID: \"6ff40a73-e740-4c46-9240-d5dba77aeefc\") " pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.602924 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.602901    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xtgt\" (UniqueName: \"kubernetes.io/projected/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-kube-api-access-5xtgt\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:19.692376 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692338    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-ovn-node-metrics-cert\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.692376 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-ovnkube-script-lib\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.692646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692407    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-cni-bin\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.692646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-cni-netd\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.692646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692459    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-systemd-units\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.692646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-var-lib-openvswitch\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.692646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692519    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5183a6f1-0901-4208-b85a-8fab51402e3e-agent-certs\") pod \"konnectivity-agent-b988b\" (UID: \"5183a6f1-0901-4208-b85a-8fab51402e3e\") " pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:19.692646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr999\" (UniqueName: \"kubernetes.io/projected/b098ac18-b58d-4a14-af63-7b1973beb36b-kube-api-access-zr999\") pod \"node-resolver-f4xh5\" (UID: \"b098ac18-b58d-4a14-af63-7b1973beb36b\") " pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.692646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-systemd-units\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.692646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692533    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-cni-netd\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.692646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-var-lib-openvswitch\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.692646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692612    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-kubelet\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-run-openvswitch\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692693    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-kubelet\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692536    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-cni-bin\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692706    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-run-openvswitch\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-env-overrides\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpmlr\" (UniqueName: \"kubernetes.io/projected/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-kube-api-access-lpmlr\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692805    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b098ac18-b58d-4a14-af63-7b1973beb36b-tmp-dir\") pod \"node-resolver-f4xh5\" (UID: \"b098ac18-b58d-4a14-af63-7b1973beb36b\") " pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-slash\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692860    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-run-ovn-kubernetes\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692888    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-run-netns\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-run-ovn\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692933    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692952    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b098ac18-b58d-4a14-af63-7b1973beb36b-hosts-file\") pod \"node-resolver-f4xh5\" (UID: \"b098ac18-b58d-4a14-af63-7b1973beb36b\") " pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.692953    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-slash\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693017    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-run-netns\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693041    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-ovnkube-script-lib\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.693132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-run-ovn-kubernetes\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693097    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b098ac18-b58d-4a14-af63-7b1973beb36b-hosts-file\") pod \"node-resolver-f4xh5\" (UID: \"b098ac18-b58d-4a14-af63-7b1973beb36b\") " pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693101    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-run-ovn\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693121    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693129    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-etc-openvswitch\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-run-systemd\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693181    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5183a6f1-0901-4208-b85a-8fab51402e3e-konnectivity-ca\") pod \"konnectivity-agent-b988b\" (UID: \"5183a6f1-0901-4208-b85a-8fab51402e3e\") " pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-log-socket\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693180    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-env-overrides\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693231    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-run-systemd\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693253    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-node-log\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-etc-openvswitch\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693276    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-ovnkube-config\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-log-socket\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b098ac18-b58d-4a14-af63-7b1973beb36b-tmp-dir\") pod \"node-resolver-f4xh5\" (UID: \"b098ac18-b58d-4a14-af63-7b1973beb36b\") " pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693342    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-node-log\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693809    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5183a6f1-0901-4208-b85a-8fab51402e3e-konnectivity-ca\") pod \"konnectivity-agent-b988b\" (UID: \"5183a6f1-0901-4208-b85a-8fab51402e3e\") " pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:19.694002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.693812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-ovnkube-config\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.695583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.695560    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5183a6f1-0901-4208-b85a-8fab51402e3e-agent-certs\") pod \"konnectivity-agent-b988b\" (UID: \"5183a6f1-0901-4208-b85a-8fab51402e3e\") " pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:19.695711 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.695693    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-ovn-node-metrics-cert\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.700567 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.700546    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpmlr\" (UniqueName: \"kubernetes.io/projected/c3e2aa5d-4161-4eef-aaa4-5ce5655d4530-kube-api-access-lpmlr\") pod \"ovnkube-node-p8p68\" (UID: \"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530\") " pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.700567 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.700557    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr999\" (UniqueName: \"kubernetes.io/projected/b098ac18-b58d-4a14-af63-7b1973beb36b-kube-api-access-zr999\") pod \"node-resolver-f4xh5\" (UID: \"b098ac18-b58d-4a14-af63-7b1973beb36b\") " pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.783861 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.783774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j5hzk"
Jul 12 18:25:19.794376 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.794350    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q9sm4"
Jul 12 18:25:19.803365 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.803340    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr"
Jul 12 18:25:19.810151 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.810129    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rxs9c"
Jul 12 18:25:19.816726 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.816706    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gx47v"
Jul 12 18:25:19.824527 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.824507    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vqgmh"
Jul 12 18:25:19.831046 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.831027    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:19.837695 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.837671    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:19.842333 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.842315    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f4xh5"
Jul 12 18:25:19.867367 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:19.867335    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 18:25:20.097481 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.097380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmscs\" (UniqueName: \"kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs\") pod \"network-check-target-q77wh\" (UID: \"d11c8121-7766-4973-9398-0ed4c73fd2e3\") " pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:20.097481 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.097451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:20.097694 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.097507    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:20.097694 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:20.097562    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:20.097694 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:20.097620    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:20.097694 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:20.097630    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret podName:adc887e4-8e0a-4ed3-9978-792e0a1573bb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:21.097615108 +0000 UTC m=+3.980965361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret") pod "global-pull-secret-syncer-4kw4m" (UID: "adc887e4-8e0a-4ed3-9978-792e0a1573bb") : object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:20.097694 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:20.097562    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 18:25:20.097694 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:20.097672    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs podName:77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:21.09765611 +0000 UTC m=+3.981006368 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs") pod "network-metrics-daemon-cbzdr" (UID: "77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:20.097694 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:20.097694    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 18:25:20.097977 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:20.097712    2562 projected.go:196] Error preparing data for projected volume kube-api-access-zmscs for pod openshift-network-diagnostics/network-check-target-q77wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:20.097977 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:20.097758    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs podName:d11c8121-7766-4973-9398-0ed4c73fd2e3 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:21.097744435 +0000 UTC m=+3.981094692 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zmscs" (UniqueName: "kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs") pod "network-check-target-q77wh" (UID: "d11c8121-7766-4973-9398-0ed4c73fd2e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:20.497338 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:20.497301    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f73fd2b_9f51_44df_98cb_672c606fc32e.slice/crio-999d520691d52d1b2fb8ed2225bba7c6a3825aec2fddda3cef4a098c1e52248f WatchSource:0}: Error finding container 999d520691d52d1b2fb8ed2225bba7c6a3825aec2fddda3cef4a098c1e52248f: Status 404 returned error can't find the container with id 999d520691d52d1b2fb8ed2225bba7c6a3825aec2fddda3cef4a098c1e52248f
Jul 12 18:25:20.498760 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:20.498703    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38c69e27_be45_4ff9_b58d_906334dd14ae.slice/crio-c9f1d6b26edaf7f3fd165b53873790c17cd3229172da8f0763f77f66849d07ff WatchSource:0}: Error finding container c9f1d6b26edaf7f3fd165b53873790c17cd3229172da8f0763f77f66849d07ff: Status 404 returned error can't find the container with id c9f1d6b26edaf7f3fd165b53873790c17cd3229172da8f0763f77f66849d07ff
Jul 12 18:25:20.499754 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:20.499717    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d8ff9f2_94d1_4aea_a9ea_16a41760f6a2.slice/crio-cd43054042396278b90ee1e50db6298d4dc153ab287dd4e5d1e489f7ad26c2b5 WatchSource:0}: Error finding container cd43054042396278b90ee1e50db6298d4dc153ab287dd4e5d1e489f7ad26c2b5: Status 404 returned error can't find the container with id cd43054042396278b90ee1e50db6298d4dc153ab287dd4e5d1e489f7ad26c2b5
Jul 12 18:25:20.504714 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:20.504691    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5183a6f1_0901_4208_b85a_8fab51402e3e.slice/crio-680e233130f4b525acf00b230a75e6a8da5a116294b24c7f6e217e6f3635338f WatchSource:0}: Error finding container 680e233130f4b525acf00b230a75e6a8da5a116294b24c7f6e217e6f3635338f: Status 404 returned error can't find the container with id 680e233130f4b525acf00b230a75e6a8da5a116294b24c7f6e217e6f3635338f
Jul 12 18:25:20.505646 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:20.505626    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb098ac18_b58d_4a14_af63_7b1973beb36b.slice/crio-f4b6750686d6e4670b257f6b72edc3d2d6af365b0079e9e2df060cd34139658e WatchSource:0}: Error finding container f4b6750686d6e4670b257f6b72edc3d2d6af365b0079e9e2df060cd34139658e: Status 404 returned error can't find the container with id f4b6750686d6e4670b257f6b72edc3d2d6af365b0079e9e2df060cd34139658e
Jul 12 18:25:20.506010 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:20.505960    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99149170_b408_430b_899c_f77f30a7f163.slice/crio-c2273dded42487d1bf4de6c8a6d8b33e153793cbc30b361b08f7f082c66bb6bc WatchSource:0}: Error finding container c2273dded42487d1bf4de6c8a6d8b33e153793cbc30b361b08f7f082c66bb6bc: Status 404 returned error can't find the container with id c2273dded42487d1bf4de6c8a6d8b33e153793cbc30b361b08f7f082c66bb6bc
Jul 12 18:25:20.508017 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:20.507533    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddec7786a_3b72_4a67_9aea_8d1030b7cbab.slice/crio-185350af98edfe5f54ede99c9bf621e7967284e56531a57c9d787788b983b696 WatchSource:0}: Error finding container 185350af98edfe5f54ede99c9bf621e7967284e56531a57c9d787788b983b696: Status 404 returned error can't find the container with id 185350af98edfe5f54ede99c9bf621e7967284e56531a57c9d787788b983b696
Jul 12 18:25:20.508280 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:20.508238    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e2aa5d_4161_4eef_aaa4_5ce5655d4530.slice/crio-53815ef27393e9c2d3a6c9b7ee56f78b51f0ba28bbb2106c0e8db62c87b07a0c WatchSource:0}: Error finding container 53815ef27393e9c2d3a6c9b7ee56f78b51f0ba28bbb2106c0e8db62c87b07a0c: Status 404 returned error can't find the container with id 53815ef27393e9c2d3a6c9b7ee56f78b51f0ba28bbb2106c0e8db62c87b07a0c
Jul 12 18:25:20.514000 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.513959    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 18:20:18 +0000 UTC" deadline="2028-02-25 18:18:37.812177634 +0000 UTC"
Jul 12 18:25:20.514000 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.513999    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14231h53m17.29818157s"
Jul 12 18:25:20.606898 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.606615    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal" event={"ID":"2a48c8d39a9521700ee4336712fb4822","Type":"ContainerStarted","Data":"bf0c5753b5963d2dc7bef7be1e8f2e270878a20fa334a319dbfaa68116b944b7"}
Jul 12 18:25:20.607905 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.607880    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" event={"ID":"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530","Type":"ContainerStarted","Data":"53815ef27393e9c2d3a6c9b7ee56f78b51f0ba28bbb2106c0e8db62c87b07a0c"}
Jul 12 18:25:20.609114 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.609094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f4xh5" event={"ID":"b098ac18-b58d-4a14-af63-7b1973beb36b","Type":"ContainerStarted","Data":"f4b6750686d6e4670b257f6b72edc3d2d6af365b0079e9e2df060cd34139658e"}
Jul 12 18:25:20.610176 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.610154    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b988b" event={"ID":"5183a6f1-0901-4208-b85a-8fab51402e3e","Type":"ContainerStarted","Data":"680e233130f4b525acf00b230a75e6a8da5a116294b24c7f6e217e6f3635338f"}
Jul 12 18:25:20.611371 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.611329    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j5hzk" event={"ID":"38c69e27-be45-4ff9-b58d-906334dd14ae","Type":"ContainerStarted","Data":"c9f1d6b26edaf7f3fd165b53873790c17cd3229172da8f0763f77f66849d07ff"}
Jul 12 18:25:20.612410 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.612386    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rxs9c" event={"ID":"6f73fd2b-9f51-44df-98cb-672c606fc32e","Type":"ContainerStarted","Data":"999d520691d52d1b2fb8ed2225bba7c6a3825aec2fddda3cef4a098c1e52248f"}
Jul 12 18:25:20.613536 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.613482    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gx47v" event={"ID":"dec7786a-3b72-4a67-9aea-8d1030b7cbab","Type":"ContainerStarted","Data":"185350af98edfe5f54ede99c9bf621e7967284e56531a57c9d787788b983b696"}
Jul 12 18:25:20.614745 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.614721    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vqgmh" event={"ID":"99149170-b408-430b-899c-f77f30a7f163","Type":"ContainerStarted","Data":"c2273dded42487d1bf4de6c8a6d8b33e153793cbc30b361b08f7f082c66bb6bc"}
Jul 12 18:25:20.615863 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.615828    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr" event={"ID":"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2","Type":"ContainerStarted","Data":"cd43054042396278b90ee1e50db6298d4dc153ab287dd4e5d1e489f7ad26c2b5"}
Jul 12 18:25:20.616908 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.616888    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q9sm4" event={"ID":"6ff40a73-e740-4c46-9240-d5dba77aeefc","Type":"ContainerStarted","Data":"33da9be9805ed628530448d06d4d97b6ef0ae692a5056b2a87a5e454bacd26ed"}
Jul 12 18:25:20.620102 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:20.620066    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-164.ec2.internal" podStartSLOduration=2.620053178 podStartE2EDuration="2.620053178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:20.619978281 +0000 UTC m=+3.503328557" watchObservedRunningTime="2026-07-12 18:25:20.620053178 +0000 UTC m=+3.503403452"
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:21.104531    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:21.104591    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:21.104648    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmscs\" (UniqueName: \"kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs\") pod \"network-check-target-q77wh\" (UID: \"d11c8121-7766-4973-9398-0ed4c73fd2e3\") " pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.104797    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.104815    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.104827    2562 projected.go:196] Error preparing data for projected volume kube-api-access-zmscs for pod openshift-network-diagnostics/network-check-target-q77wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.104884    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs podName:d11c8121-7766-4973-9398-0ed4c73fd2e3 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:23.104864652 +0000 UTC m=+5.988214907 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zmscs" (UniqueName: "kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs") pod "network-check-target-q77wh" (UID: "d11c8121-7766-4973-9398-0ed4c73fd2e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.105271    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.105321    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret podName:adc887e4-8e0a-4ed3-9978-792e0a1573bb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:23.105304912 +0000 UTC m=+5.988655169 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret") pod "global-pull-secret-syncer-4kw4m" (UID: "adc887e4-8e0a-4ed3-9978-792e0a1573bb") : object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.105375    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:21.105437 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.105405    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs podName:77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:23.105395378 +0000 UTC m=+5.988745634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs") pod "network-metrics-daemon-cbzdr" (UID: "77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:21.599649 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:21.598831    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:21.599649 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.598951    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:21.599649 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:21.599349    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:21.599649 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.599448    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:21.599649 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:21.599534    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:21.599649 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:21.599611    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:21.627134 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:21.627101    2562 generic.go:356] "Generic (PLEG): container finished" podID="1d30db9f1d57f95317b299b4a177ebe3" containerID="98ad22a8acc8d80fd66f9db8e290a0cf1b48485cf2e2e485a1be0e5c8990cab2" exitCode=0
Jul 12 18:25:21.628161 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:21.628056    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal" event={"ID":"1d30db9f1d57f95317b299b4a177ebe3","Type":"ContainerDied","Data":"98ad22a8acc8d80fd66f9db8e290a0cf1b48485cf2e2e485a1be0e5c8990cab2"}
Jul 12 18:25:22.657046 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:22.656979    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal" event={"ID":"1d30db9f1d57f95317b299b4a177ebe3","Type":"ContainerStarted","Data":"eadfa5d658a1b706ddc0ed6e48447cc8d66d08cdf3ceab83ce7d890bae76aaf1"}
Jul 12 18:25:23.123286 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:23.123191    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:23.123286 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:23.123261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:23.123515 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:23.123319    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmscs\" (UniqueName: \"kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs\") pod \"network-check-target-q77wh\" (UID: \"d11c8121-7766-4973-9398-0ed4c73fd2e3\") " pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:23.123515 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.123496    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 18:25:23.123515 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.123514    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 18:25:23.123683 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.123526    2562 projected.go:196] Error preparing data for projected volume kube-api-access-zmscs for pod openshift-network-diagnostics/network-check-target-q77wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:23.123683 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.123583    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs podName:d11c8121-7766-4973-9398-0ed4c73fd2e3 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:27.123564947 +0000 UTC m=+10.006915207 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zmscs" (UniqueName: "kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs") pod "network-check-target-q77wh" (UID: "d11c8121-7766-4973-9398-0ed4c73fd2e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:23.124046 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.124026    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:23.124125 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.124088    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret podName:adc887e4-8e0a-4ed3-9978-792e0a1573bb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:27.124071232 +0000 UTC m=+10.007421493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret") pod "global-pull-secret-syncer-4kw4m" (UID: "adc887e4-8e0a-4ed3-9978-792e0a1573bb") : object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:23.124185 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.124146    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:23.124185 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.124179    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs podName:77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:27.124166907 +0000 UTC m=+10.007517165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs") pod "network-metrics-daemon-cbzdr" (UID: "77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:23.597664 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:23.597125    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:23.597664 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:23.597125    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:23.597664 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.597368    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:23.597664 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.597264    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:23.597664 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:23.597469    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:23.597664 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:23.597550    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:25.596744 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:25.596678    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:25.597170 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:25.596805    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:25.597170 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:25.596959    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:25.597170 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:25.597006    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:25.597170 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:25.597096    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:25.597317 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:25.597192    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:27.160164 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:27.160121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmscs\" (UniqueName: \"kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs\") pod \"network-check-target-q77wh\" (UID: \"d11c8121-7766-4973-9398-0ed4c73fd2e3\") " pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:27.160720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:27.160202    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:27.160720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:27.160246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:27.160720 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.160408    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:27.160720 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.160489    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs podName:77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:35.160451455 +0000 UTC m=+18.043801730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs") pod "network-metrics-daemon-cbzdr" (UID: "77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:27.161051 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.160937    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 18:25:27.161051 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.160957    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 18:25:27.161051 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.160969    2562 projected.go:196] Error preparing data for projected volume kube-api-access-zmscs for pod openshift-network-diagnostics/network-check-target-q77wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:27.161051 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.161018    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs podName:d11c8121-7766-4973-9398-0ed4c73fd2e3 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:35.161002088 +0000 UTC m=+18.044352362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zmscs" (UniqueName: "kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs") pod "network-check-target-q77wh" (UID: "d11c8121-7766-4973-9398-0ed4c73fd2e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:27.161294 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.161085    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:27.161294 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.161125    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret podName:adc887e4-8e0a-4ed3-9978-792e0a1573bb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:35.161113231 +0000 UTC m=+18.044463485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret") pod "global-pull-secret-syncer-4kw4m" (UID: "adc887e4-8e0a-4ed3-9978-792e0a1573bb") : object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:27.598212 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:27.598113    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:27.598450 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.598230    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:27.598450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:27.598280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:27.598450 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.598425    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:27.598639 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:27.598473    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:27.598639 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:27.598560    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:29.597503 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:29.597424    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:29.597859 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:29.597564    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:29.597859 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:29.597582    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:29.597859 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:29.597699    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:29.597859 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:29.597751    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:29.597859 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:29.597829    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:31.596754 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:31.596724    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:31.597183 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:31.596719    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:31.597183 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:31.596839    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:31.597183 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:31.596720    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:31.597183 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:31.596923    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:31.597183 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:31.597066    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:33.597151 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:33.597079    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:33.597615 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:33.597094    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:33.597615 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:33.597208    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:33.597615 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:33.597212    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:33.597615 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:33.597293    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:33.597615 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:33.597361    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:35.219879 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:35.219833    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:35.220372 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:35.219912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmscs\" (UniqueName: \"kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs\") pod \"network-check-target-q77wh\" (UID: \"d11c8121-7766-4973-9398-0ed4c73fd2e3\") " pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:35.220372 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:35.219948    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:35.220372 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.220001    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:35.220372 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.220045    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:35.220372 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.220061    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 18:25:35.220372 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.220079    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 18:25:35.220372 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.220092    2562 projected.go:196] Error preparing data for projected volume kube-api-access-zmscs for pod openshift-network-diagnostics/network-check-target-q77wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:35.220372 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.220079    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs podName:77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:51.220058406 +0000 UTC m=+34.103408661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs") pod "network-metrics-daemon-cbzdr" (UID: "77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:35.220372 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.220152    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret podName:adc887e4-8e0a-4ed3-9978-792e0a1573bb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:51.220141039 +0000 UTC m=+34.103491294 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret") pod "global-pull-secret-syncer-4kw4m" (UID: "adc887e4-8e0a-4ed3-9978-792e0a1573bb") : object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:35.220372 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.220171    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs podName:d11c8121-7766-4973-9398-0ed4c73fd2e3 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:51.220163099 +0000 UTC m=+34.103513350 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zmscs" (UniqueName: "kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs") pod "network-check-target-q77wh" (UID: "d11c8121-7766-4973-9398-0ed4c73fd2e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:35.597119 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:35.597006    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:35.597119 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:35.597019    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:35.597317 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.597134    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:35.597317 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:35.597152    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:35.597317 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.597280    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:35.597531 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:35.597385    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:37.602474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:37.602390    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:37.602474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:37.602427    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:37.602952 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:37.602517    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:37.602952 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:37.602580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:37.602952 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:37.602666    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:37.602952 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:37.602759    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:39.601259 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.601107    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:39.601721 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:39.601351    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:39.601721 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.601151    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:39.601721 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.601172    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:39.601721 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:39.601451    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:39.601721 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:39.601533    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:39.697665 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.697586    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gx47v" event={"ID":"dec7786a-3b72-4a67-9aea-8d1030b7cbab","Type":"ContainerStarted","Data":"5b57fed97d79f68777d722683f36347f5ad96960c9bd9b813783200313ac7381"}
Jul 12 18:25:39.699507 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.699477    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr" event={"ID":"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2","Type":"ContainerStarted","Data":"54a7a7fc770d5c7211fe1ce68f0a452a0f56462d022d09264072c705ebc1d20f"}
Jul 12 18:25:39.701259 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.701233    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q9sm4" event={"ID":"6ff40a73-e740-4c46-9240-d5dba77aeefc","Type":"ContainerStarted","Data":"7b0701580d22a4e6cd7cc145cee28bf6687295d890bd581143753106c2b92d0e"}
Jul 12 18:25:39.703189 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.703107    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f4xh5" event={"ID":"b098ac18-b58d-4a14-af63-7b1973beb36b","Type":"ContainerStarted","Data":"b4572cfa01e6361519c5615296cf944935ca719ac9efa887ef35614efd689357"}
Jul 12 18:25:39.705095 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.705064    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b988b" event={"ID":"5183a6f1-0901-4208-b85a-8fab51402e3e","Type":"ContainerStarted","Data":"326702d0efd5062ed6d05d337dabbdf87210c7a621edd94913d111ff7eae532f"}
Jul 12 18:25:39.707025 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.706974    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j5hzk" event={"ID":"38c69e27-be45-4ff9-b58d-906334dd14ae","Type":"ContainerStarted","Data":"1a1cb837bfa5a962c1ed4130ca1a76d589a7d97d097ba0c3729b97248015ea05"}
Jul 12 18:25:39.708670 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.708644    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rxs9c" event={"ID":"6f73fd2b-9f51-44df-98cb-672c606fc32e","Type":"ContainerStarted","Data":"2de09cd20b102fa5f97c4f6be1cc14c88633d01d618e5a9d25055f0f7010b2b5"}
Jul 12 18:25:39.720934 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.720891    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-164.ec2.internal" podStartSLOduration=21.720873627 podStartE2EDuration="21.720873627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:22.673504891 +0000 UTC m=+5.556855167" watchObservedRunningTime="2026-07-12 18:25:39.720873627 +0000 UTC m=+22.604223900"
Jul 12 18:25:39.734825 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.734785    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j5hzk" podStartSLOduration=3.876626913 podStartE2EDuration="22.73477223s" totalImagesPullingTime="18.858145317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:39.734485429 +0000 UTC m=+22.617835694" watchObservedRunningTime="2026-07-12 18:25:39.73477223 +0000 UTC m=+22.618122504"
Jul 12 18:25:39.749892 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.749854    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b988b" podStartSLOduration=12.287305469 podStartE2EDuration="22.74983943s" totalImagesPullingTime="10.462533961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:39.749594013 +0000 UTC m=+22.632944286" watchObservedRunningTime="2026-07-12 18:25:39.74983943 +0000 UTC m=+22.633189704"
Jul 12 18:25:39.765664 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.765624    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-f4xh5" podStartSLOduration=3.915327776 podStartE2EDuration="22.765608886s" totalImagesPullingTime="18.85028111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:39.765290008 +0000 UTC m=+22.648640283" watchObservedRunningTime="2026-07-12 18:25:39.765608886 +0000 UTC m=+22.648959159"
Jul 12 18:25:39.781603 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.781566    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q9sm4" podStartSLOduration=3.900644653 podStartE2EDuration="22.781552013s" totalImagesPullingTime="18.88090736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:39.781202179 +0000 UTC m=+22.664552457" watchObservedRunningTime="2026-07-12 18:25:39.781552013 +0000 UTC m=+22.664902287"
Jul 12 18:25:39.794281 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:39.794237    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rxs9c" podStartSLOduration=3.935637378 podStartE2EDuration="22.794224067s" totalImagesPullingTime="18.858586689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:39.79410693 +0000 UTC m=+22.677457204" watchObservedRunningTime="2026-07-12 18:25:39.794224067 +0000 UTC m=+22.677574341"
Jul 12 18:25:40.713272 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.713106    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:25:40.713803 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.713567    2562 generic.go:356] "Generic (PLEG): container finished" podID="c3e2aa5d-4161-4eef-aaa4-5ce5655d4530" containerID="411e69432909a72524f07dd6f87e07107dd9d2087a24daa833bcf42f5625c439" exitCode=1
Jul 12 18:25:40.713803 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.713635    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" event={"ID":"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530","Type":"ContainerStarted","Data":"3d4d94d1ad83ddbfb69546e5e492fc0fc911866691809785071e67ee843db7c6"}
Jul 12 18:25:40.713803 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.713773    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" event={"ID":"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530","Type":"ContainerStarted","Data":"031a38ee98cbd1aeb52080597f547f5bfc6b579032ad3f97da8564d74064d11a"}
Jul 12 18:25:40.714048 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.713829    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" event={"ID":"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530","Type":"ContainerStarted","Data":"d8b4f5c6ce02503eb4d17d4976f458d0df59896f7b648600e74c0b3d3d3ff9af"}
Jul 12 18:25:40.714048 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.713886    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" event={"ID":"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530","Type":"ContainerStarted","Data":"a122f7e8650380311935d7fbfe4061069fe745a2bb491eb20bf21d025412a82c"}
Jul 12 18:25:40.714048 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.713915    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" event={"ID":"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530","Type":"ContainerDied","Data":"411e69432909a72524f07dd6f87e07107dd9d2087a24daa833bcf42f5625c439"}
Jul 12 18:25:40.714048 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.713943    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" event={"ID":"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530","Type":"ContainerStarted","Data":"3e64e2944c317e61a96fbe08072db136c66959a2bffa780b8b491b76609c5f1e"}
Jul 12 18:25:40.714957 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.714939    2562 generic.go:356] "Generic (PLEG): container finished" podID="dec7786a-3b72-4a67-9aea-8d1030b7cbab" containerID="5b57fed97d79f68777d722683f36347f5ad96960c9bd9b813783200313ac7381" exitCode=0
Jul 12 18:25:40.715031 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.714999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gx47v" event={"ID":"dec7786a-3b72-4a67-9aea-8d1030b7cbab","Type":"ContainerDied","Data":"5b57fed97d79f68777d722683f36347f5ad96960c9bd9b813783200313ac7381"}
Jul 12 18:25:40.716314 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.716291    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vqgmh" event={"ID":"99149170-b408-430b-899c-f77f30a7f163","Type":"ContainerStarted","Data":"c899639ea3679d730cfc0288b014ad678638b61cdc7859e4765542f13faa65ca"}
Jul 12 18:25:40.985905 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:40.985877    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 12 18:25:41.546048 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:41.545932    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T18:25:40.985902834Z","UUID":"d68cff96-1fcf-40fb-b64a-f4b3ef5ebe60","Handler":null,"Name":"","Endpoint":""}
Jul 12 18:25:41.547786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:41.547748    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 12 18:25:41.547786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:41.547785    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 12 18:25:41.597329 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:41.597294    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:41.597329 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:41.597318    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:41.597581 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:41.597307    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:41.597581 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:41.597467    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:41.597581 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:41.597535    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:41.597732 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:41.597623    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:41.721452 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:41.721410    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr" event={"ID":"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2","Type":"ContainerStarted","Data":"548d7fcaba27cb27acf81f9ca62567a83f4f03b40de46693ed207f4134751410"}
Jul 12 18:25:42.726807 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:42.726601    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:25:42.727203 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:42.727144    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" event={"ID":"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530","Type":"ContainerStarted","Data":"78ba7d9ca70e1499a541ae78de5a9d73f95a9c93dd465a0ad37a73e5ddcdd76d"}
Jul 12 18:25:42.729550 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:42.729523    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr" event={"ID":"6d8ff9f2-94d1-4aea-a9ea-16a41760f6a2","Type":"ContainerStarted","Data":"f9ffaa3d664c546c6f9aa14ceeb95532b060654ebb9a417aa6ca91b75df2c4f2"}
Jul 12 18:25:42.747660 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:42.747616    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgpbr" podStartSLOduration=4.464731178 podStartE2EDuration="25.747603645s" totalImagesPullingTime="21.282872467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:42.747171042 +0000 UTC m=+25.630521320" watchObservedRunningTime="2026-07-12 18:25:42.747603645 +0000 UTC m=+25.630953923"
Jul 12 18:25:42.747932 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:42.747899    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vqgmh" podStartSLOduration=6.8990696929999995 podStartE2EDuration="25.747894595s" totalImagesPullingTime="18.848824902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:40.750710645 +0000 UTC m=+23.634060918" watchObservedRunningTime="2026-07-12 18:25:42.747894595 +0000 UTC m=+25.631244869"
Jul 12 18:25:43.060793 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:43.060708    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:43.061355 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:43.061332    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:43.597084 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:43.597049    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:43.597280 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:43.597049    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:43.597280 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:43.597176    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:43.597280 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:43.597188    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:43.597431 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:43.597308    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:43.597494 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:43.597438    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:44.724027 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:44.723974    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:44.803558 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:44.803528    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:45.596796 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.596578    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:45.596945 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.596629    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:45.596945 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:45.596837    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:45.596945 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.596649    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:45.597118 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:45.596963    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:45.597118 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:45.596903    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:45.739639 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.739609    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:25:45.740124 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.739937    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" event={"ID":"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530","Type":"ContainerStarted","Data":"31978ac3c534f38920d7d84485efca59c2d43e53fdb251178385a9314cdfadfd"}
Jul 12 18:25:45.740124 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.740091    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:45.740249 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.740157    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:45.740249 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.740209    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:45.740378 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.740363    2562 scope.go:117] "RemoveContainer" containerID="411e69432909a72524f07dd6f87e07107dd9d2087a24daa833bcf42f5625c439"
Jul 12 18:25:45.742409 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.742382    2562 generic.go:356] "Generic (PLEG): container finished" podID="dec7786a-3b72-4a67-9aea-8d1030b7cbab" containerID="70fbd44c191144c456987d5fcd8d0df16d2ec31a3fb69f2d5cd0a60e8815dc8b" exitCode=0
Jul 12 18:25:45.742539 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.742432    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gx47v" event={"ID":"dec7786a-3b72-4a67-9aea-8d1030b7cbab","Type":"ContainerDied","Data":"70fbd44c191144c456987d5fcd8d0df16d2ec31a3fb69f2d5cd0a60e8815dc8b"}
Jul 12 18:25:45.757300 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.757223    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:45.757420 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:45.757408    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:25:46.748321 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:46.748298    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:25:46.748761 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:46.748728    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" event={"ID":"c3e2aa5d-4161-4eef-aaa4-5ce5655d4530","Type":"ContainerStarted","Data":"d6fe83e56e7cf233a03debd49f4695a1b08148a2b3cdec82c0dd8deca96040da"}
Jul 12 18:25:46.777744 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:46.777690    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68" podStartSLOduration=10.528705474 podStartE2EDuration="29.77766785s" totalImagesPullingTime="19.248962376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:46.776617655 +0000 UTC m=+29.659967932" watchObservedRunningTime="2026-07-12 18:25:46.77766785 +0000 UTC m=+29.661018125"
Jul 12 18:25:47.086623 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:47.086401    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4kw4m"]
Jul 12 18:25:47.086776 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:47.086665    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:47.086776 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:47.086747    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:47.089436 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:47.089410    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q77wh"]
Jul 12 18:25:47.089582 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:47.089539    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:47.089654 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:47.089638    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:47.090141 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:47.090117    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cbzdr"]
Jul 12 18:25:47.090258 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:47.090232    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:47.090353 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:47.090335    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:47.752744 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:47.752711    2562 generic.go:356] "Generic (PLEG): container finished" podID="dec7786a-3b72-4a67-9aea-8d1030b7cbab" containerID="9ecd5300c5bc802b1900e1d0ae1d3911b9dbb90ccd971763111e2564a9b62a99" exitCode=0
Jul 12 18:25:47.753186 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:47.752778    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gx47v" event={"ID":"dec7786a-3b72-4a67-9aea-8d1030b7cbab","Type":"ContainerDied","Data":"9ecd5300c5bc802b1900e1d0ae1d3911b9dbb90ccd971763111e2564a9b62a99"}
Jul 12 18:25:48.481600 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:48.481567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:48.551043 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:48.551005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:48.596805 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:48.596773    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:48.596805 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:48.596806    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:48.597002 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:48.596817    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:48.597002 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:48.596900    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:48.597072 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:48.597021    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:48.597116 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:48.597091    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:48.689981 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:48.689951    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:48.750436 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:48.750416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:49.427413 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:49.427343    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:49.428040 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:49.427505    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 12 18:25:49.428274 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:49.428253    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b988b"
Jul 12 18:25:49.488852 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:49.488825    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:49.759731 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:49.759653    2562 generic.go:356] "Generic (PLEG): container finished" podID="dec7786a-3b72-4a67-9aea-8d1030b7cbab" containerID="a07df040cec64c551b55719dd9626c2dc2fe1987654c486bac2a5e17a2ca4072" exitCode=0
Jul 12 18:25:49.759860 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:49.759740    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gx47v" event={"ID":"dec7786a-3b72-4a67-9aea-8d1030b7cbab","Type":"ContainerDied","Data":"a07df040cec64c551b55719dd9626c2dc2fe1987654c486bac2a5e17a2ca4072"}
Jul 12 18:25:50.530486 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:50.530426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:50.596720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:50.596684    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:50.596720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:50.596708    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:50.596720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:50.596693    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:50.596983 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:50.596812    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbzdr" podUID="77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0"
Jul 12 18:25:50.596983 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:50.596849    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4kw4m" podUID="adc887e4-8e0a-4ed3-9978-792e0a1573bb"
Jul 12 18:25:50.596983 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:50.596957    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q77wh" podUID="d11c8121-7766-4973-9398-0ed4c73fd2e3"
Jul 12 18:25:50.734150 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:50.734057    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:50.835775 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:50.835697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:51.244157 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:51.244080    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:51.244157 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:51.244141    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmscs\" (UniqueName: \"kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs\") pod \"network-check-target-q77wh\" (UID: \"d11c8121-7766-4973-9398-0ed4c73fd2e3\") " pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:51.244382 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:51.244184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:51.244382 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:51.244236    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:51.244382 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:51.244276    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:51.244382 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:51.244300    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs podName:77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:23.244283207 +0000 UTC m=+66.127633463 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs") pod "network-metrics-daemon-cbzdr" (UID: "77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 18:25:51.244382 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:51.244297    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 18:25:51.244382 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:51.244323    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret podName:adc887e4-8e0a-4ed3-9978-792e0a1573bb nodeName:}" failed. No retries permitted until 2026-07-12 18:26:23.244308945 +0000 UTC m=+66.127659199 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret") pod "global-pull-secret-syncer-4kw4m" (UID: "adc887e4-8e0a-4ed3-9978-792e0a1573bb") : object "kube-system"/"original-pull-secret" not registered
Jul 12 18:25:51.244382 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:51.244329    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 18:25:51.244382 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:51.244341    2562 projected.go:196] Error preparing data for projected volume kube-api-access-zmscs for pod openshift-network-diagnostics/network-check-target-q77wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:51.244382 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:51.244373    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs podName:d11c8121-7766-4973-9398-0ed4c73fd2e3 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:23.244364234 +0000 UTC m=+66.127714487 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zmscs" (UniqueName: "kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs") pod "network-check-target-q77wh" (UID: "d11c8121-7766-4973-9398-0ed4c73fd2e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 18:25:51.658259 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:51.658164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:51.759266 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:51.759236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:52.298590 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.298517    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:52.399475 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.399257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:52.517700 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.517655    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-164.ec2.internal" event="NodeReady"
Jul 12 18:25:52.517888 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.517833    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 12 18:25:52.553270 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.553188    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6dcbcb4984-kjk44"]
Jul 12 18:25:52.555338 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.555314    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.557830 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.557809    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 12 18:25:52.558367 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.558173    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-twcn4\""
Jul 12 18:25:52.558367 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.558186    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 12 18:25:52.558872 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.558849    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 12 18:25:52.563292 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.563271    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 12 18:25:52.564930 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.564911    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h8jls"]
Jul 12 18:25:52.566840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.566823    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.569504 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.569387    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 12 18:25:52.569504 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.569403    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7m82z\""
Jul 12 18:25:52.569796 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.569698    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 12 18:25:52.569960 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.569943    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7msr2"]
Jul 12 18:25:52.571924 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.571900    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dcbcb4984-kjk44"]
Jul 12 18:25:52.572008 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.571993    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7msr2"
Jul 12 18:25:52.574370 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.574329    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 12 18:25:52.574507 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.574360    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 12 18:25:52.574507 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.574432    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m2nlf\""
Jul 12 18:25:52.574605 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.574329    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 12 18:25:52.579069 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.579043    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h8jls"]
Jul 12 18:25:52.583059 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.582668    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7msr2"]
Jul 12 18:25:52.597218 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.597191    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:25:52.597369 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.597352    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:25:52.597466 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.597438    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:25:52.599647 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.599582    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p8prd\""
Jul 12 18:25:52.599647 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.599614    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 12 18:25:52.599830 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.599780    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 12 18:25:52.600030 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.600010    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 12 18:25:52.600140 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.600055    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gx6mz\""
Jul 12 18:25:52.600140 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.600012    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 12 18:25:52.656424 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656395    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-image-registry-private-configuration\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.656564 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cee6a773-686d-4ef4-a904-110a2826bce6-tmp-dir\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.656564 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656472    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/96c3b950-1121-433a-9eff-8dfa9a0034a0-ca-trust-extracted\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.656564 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656531    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-certificates\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.656707 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656577    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-bound-sa-token\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.656707 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656603    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5fck\" (UniqueName: \"kubernetes.io/projected/cee6a773-686d-4ef4-a904-110a2826bce6-kube-api-access-b5fck\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.656707 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656689    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-trusted-ca\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.656835 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656744    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cee6a773-686d-4ef4-a904-110a2826bce6-config-volume\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.656835 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656776    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.656835 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.656977 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656895    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-installation-pull-secrets\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.656977 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2be4bca4-2772-4c13-a24b-eebc9d61cc7d-cert\") pod \"ingress-canary-7msr2\" (UID: \"2be4bca4-2772-4c13-a24b-eebc9d61cc7d\") " pod="openshift-ingress-canary/ingress-canary-7msr2"
Jul 12 18:25:52.657049 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.656973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc2g4\" (UniqueName: \"kubernetes.io/projected/2be4bca4-2772-4c13-a24b-eebc9d61cc7d-kube-api-access-xc2g4\") pod \"ingress-canary-7msr2\" (UID: \"2be4bca4-2772-4c13-a24b-eebc9d61cc7d\") " pod="openshift-ingress-canary/ingress-canary-7msr2"
Jul 12 18:25:52.657049 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.657022    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjh2d\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-kube-api-access-jjh2d\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.738700 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.738664    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:52.757511 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757473    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/96c3b950-1121-433a-9eff-8dfa9a0034a0-ca-trust-extracted\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.757695 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757526    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-certificates\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.757695 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-bound-sa-token\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.757775 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5fck\" (UniqueName: \"kubernetes.io/projected/cee6a773-686d-4ef4-a904-110a2826bce6-kube-api-access-b5fck\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.757829 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757777    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-trusted-ca\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.757829 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cee6a773-686d-4ef4-a904-110a2826bce6-config-volume\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.757919 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757846    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.757919 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.757919 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-installation-pull-secrets\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.758039 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757928    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/96c3b950-1121-433a-9eff-8dfa9a0034a0-ca-trust-extracted\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.758039 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757936    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2be4bca4-2772-4c13-a24b-eebc9d61cc7d-cert\") pod \"ingress-canary-7msr2\" (UID: \"2be4bca4-2772-4c13-a24b-eebc9d61cc7d\") " pod="openshift-ingress-canary/ingress-canary-7msr2"
Jul 12 18:25:52.758039 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.757986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc2g4\" (UniqueName: \"kubernetes.io/projected/2be4bca4-2772-4c13-a24b-eebc9d61cc7d-kube-api-access-xc2g4\") pod \"ingress-canary-7msr2\" (UID: \"2be4bca4-2772-4c13-a24b-eebc9d61cc7d\") " pod="openshift-ingress-canary/ingress-canary-7msr2"
Jul 12 18:25:52.758039 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.758022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjh2d\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-kube-api-access-jjh2d\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.758298 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:52.758042    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 18:25:52.758298 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.758067    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-image-registry-private-configuration\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.758298 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:52.758110    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls podName:cee6a773-686d-4ef4-a904-110a2826bce6 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:53.258090311 +0000 UTC m=+36.141440571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls") pod "dns-default-h8jls" (UID: "cee6a773-686d-4ef4-a904-110a2826bce6") : secret "dns-default-metrics-tls" not found
Jul 12 18:25:52.758298 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.758130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cee6a773-686d-4ef4-a904-110a2826bce6-tmp-dir\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.758298 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:52.758207    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 18:25:52.758298 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:52.758224    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcbcb4984-kjk44: secret "image-registry-tls" not found
Jul 12 18:25:52.758298 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.758253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-certificates\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.758298 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:52.758282    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls podName:96c3b950-1121-433a-9eff-8dfa9a0034a0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:53.258266159 +0000 UTC m=+36.141616418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls") pod "image-registry-6dcbcb4984-kjk44" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0") : secret "image-registry-tls" not found
Jul 12 18:25:52.758784 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.758520    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cee6a773-686d-4ef4-a904-110a2826bce6-tmp-dir\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.759171 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.759075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-trusted-ca\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.759551 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.759515    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cee6a773-686d-4ef4-a904-110a2826bce6-config-volume\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.762581 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.762561    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-image-registry-private-configuration\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.762674 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.762564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-installation-pull-secrets\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.762674 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.762606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2be4bca4-2772-4c13-a24b-eebc9d61cc7d-cert\") pod \"ingress-canary-7msr2\" (UID: \"2be4bca4-2772-4c13-a24b-eebc9d61cc7d\") " pod="openshift-ingress-canary/ingress-canary-7msr2"
Jul 12 18:25:52.766583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.766561    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5fck\" (UniqueName: \"kubernetes.io/projected/cee6a773-686d-4ef4-a904-110a2826bce6-kube-api-access-b5fck\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:52.766857 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.766832    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-bound-sa-token\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.767400 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.767362    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc2g4\" (UniqueName: \"kubernetes.io/projected/2be4bca4-2772-4c13-a24b-eebc9d61cc7d-kube-api-access-xc2g4\") pod \"ingress-canary-7msr2\" (UID: \"2be4bca4-2772-4c13-a24b-eebc9d61cc7d\") " pod="openshift-ingress-canary/ingress-canary-7msr2"
Jul 12 18:25:52.768216 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.768170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjh2d\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-kube-api-access-jjh2d\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:52.839101 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.839018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:52.886805 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:52.886771    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7msr2"
Jul 12 18:25:53.262081 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:53.262047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:53.262081 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:53.262084    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:53.262306 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:53.262211    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 18:25:53.262306 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:53.262227    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcbcb4984-kjk44: secret "image-registry-tls" not found
Jul 12 18:25:53.262306 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:53.262284    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls podName:96c3b950-1121-433a-9eff-8dfa9a0034a0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:54.262270946 +0000 UTC m=+37.145621197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls") pod "image-registry-6dcbcb4984-kjk44" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0") : secret "image-registry-tls" not found
Jul 12 18:25:53.262306 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:53.262214    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 18:25:53.262502 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:53.262377    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls podName:cee6a773-686d-4ef4-a904-110a2826bce6 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:54.262354604 +0000 UTC m=+37.145704860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls") pod "dns-default-h8jls" (UID: "cee6a773-686d-4ef4-a904-110a2826bce6") : secret "dns-default-metrics-tls" not found
Jul 12 18:25:53.969262 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:53.969222    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7msr2"]
Jul 12 18:25:53.972785 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:53.972759    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be4bca4_2772_4c13_a24b_eebc9d61cc7d.slice/crio-1357a9c4fc6b256b494888dc3c697e890504e4f375026105e525a247209a62e0 WatchSource:0}: Error finding container 1357a9c4fc6b256b494888dc3c697e890504e4f375026105e525a247209a62e0: Status 404 returned error can't find the container with id 1357a9c4fc6b256b494888dc3c697e890504e4f375026105e525a247209a62e0
Jul 12 18:25:54.271517 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:54.271484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:54.271698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:54.271528    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:54.271698 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:54.271631    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 18:25:54.271698 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:54.271693    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 18:25:54.271812 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:54.271710    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcbcb4984-kjk44: secret "image-registry-tls" not found
Jul 12 18:25:54.271812 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:54.271695    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls podName:cee6a773-686d-4ef4-a904-110a2826bce6 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:56.271679645 +0000 UTC m=+39.155029900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls") pod "dns-default-h8jls" (UID: "cee6a773-686d-4ef4-a904-110a2826bce6") : secret "dns-default-metrics-tls" not found
Jul 12 18:25:54.271812 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:54.271767    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls podName:96c3b950-1121-433a-9eff-8dfa9a0034a0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:56.271750432 +0000 UTC m=+39.155100686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls") pod "image-registry-6dcbcb4984-kjk44" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0") : secret "image-registry-tls" not found
Jul 12 18:25:54.772685 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:54.772636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7msr2" event={"ID":"2be4bca4-2772-4c13-a24b-eebc9d61cc7d","Type":"ContainerStarted","Data":"1357a9c4fc6b256b494888dc3c697e890504e4f375026105e525a247209a62e0"}
Jul 12 18:25:54.775856 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:54.775825    2562 generic.go:356] "Generic (PLEG): container finished" podID="dec7786a-3b72-4a67-9aea-8d1030b7cbab" containerID="b1c5516bf00dd7ea59006953143d094d08e1bfec8196a915d4c5bb05612d004a" exitCode=0
Jul 12 18:25:54.776009 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:54.775864    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gx47v" event={"ID":"dec7786a-3b72-4a67-9aea-8d1030b7cbab","Type":"ContainerDied","Data":"b1c5516bf00dd7ea59006953143d094d08e1bfec8196a915d4c5bb05612d004a"}
Jul 12 18:25:55.128959 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:55.128727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:55.229787 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:55.229752    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:55.329028 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:55.328998    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:55.781882 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:55.781842    2562 generic.go:356] "Generic (PLEG): container finished" podID="dec7786a-3b72-4a67-9aea-8d1030b7cbab" containerID="a3396f466ae85667ee86918b4f838105b6eb7f2b6c43c5b5c77d7410f493ef16" exitCode=0
Jul 12 18:25:55.782104 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:55.781900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gx47v" event={"ID":"dec7786a-3b72-4a67-9aea-8d1030b7cbab","Type":"ContainerDied","Data":"a3396f466ae85667ee86918b4f838105b6eb7f2b6c43c5b5c77d7410f493ef16"}
Jul 12 18:25:56.289502 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.289469    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:25:56.289942 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.289510    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:25:56.289942 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:56.289601    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 18:25:56.289942 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:56.289661    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 18:25:56.289942 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:56.289675    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcbcb4984-kjk44: secret "image-registry-tls" not found
Jul 12 18:25:56.289942 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:56.289666    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls podName:cee6a773-686d-4ef4-a904-110a2826bce6 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:00.289650447 +0000 UTC m=+43.173000704 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls") pod "dns-default-h8jls" (UID: "cee6a773-686d-4ef4-a904-110a2826bce6") : secret "dns-default-metrics-tls" not found
Jul 12 18:25:56.289942 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:56.289737    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls podName:96c3b950-1121-433a-9eff-8dfa9a0034a0 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:00.289720087 +0000 UTC m=+43.173070345 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls") pod "image-registry-6dcbcb4984-kjk44" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0") : secret "image-registry-tls" not found
Jul 12 18:25:56.786720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.786682    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7msr2" event={"ID":"2be4bca4-2772-4c13-a24b-eebc9d61cc7d","Type":"ContainerStarted","Data":"c91f31f2090b7dcac44464de74be7780327367df721be5d61f5ee7a2b7781f0e"}
Jul 12 18:25:56.790119 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.790094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gx47v" event={"ID":"dec7786a-3b72-4a67-9aea-8d1030b7cbab","Type":"ContainerStarted","Data":"dcd70a3bc92c4c3c7d415ea670f63420f69e58edf0a5e6c30aa13b1300205bc9"}
Jul 12 18:25:56.866230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.866168    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7msr2" podStartSLOduration=2.210816897 podStartE2EDuration="4.866147554s" totalImagesPullingTime="2.655330657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:56.821957457 +0000 UTC m=+39.705307732" watchObservedRunningTime="2026-07-12 18:25:56.866147554 +0000 UTC m=+39.749497828"
Jul 12 18:25:56.866576 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.866559    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-znnz5"]
Jul 12 18:25:56.884483 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.884394    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:56.885603 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.885552    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gx47v" podStartSLOduration=10.561217053 podStartE2EDuration="39.885535592s" totalImagesPullingTime="29.324318539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:56.876769133 +0000 UTC m=+39.760119406" watchObservedRunningTime="2026-07-12 18:25:56.885535592 +0000 UTC m=+39.768885869"
Jul 12 18:25:56.886266 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.886244    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-znnz5"]
Jul 12 18:25:56.887272 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.887123    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 12 18:25:56.887272 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.887141    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 12 18:25:56.887272 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.887156    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 12 18:25:56.887272 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.887196    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 12 18:25:56.887272 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.887262    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4qrm6\""
Jul 12 18:25:56.896708 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.896672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:56.993947 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.993906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8476a109-14a2-44ba-b1b7-69d80207150f-crio-socket\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:56.993947 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.993946    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:56.994205 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.993969    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:56.994205 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.993987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jkz6\" (UniqueName: \"kubernetes.io/projected/8476a109-14a2-44ba-b1b7-69d80207150f-kube-api-access-4jkz6\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:56.994205 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.994147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8476a109-14a2-44ba-b1b7-69d80207150f-data-volume\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:56.997214 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:56.997191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:57.088658 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.088614    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:57.094694 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.094673    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jkz6\" (UniqueName: \"kubernetes.io/projected/8476a109-14a2-44ba-b1b7-69d80207150f-kube-api-access-4jkz6\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:57.094781 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.094751    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8476a109-14a2-44ba-b1b7-69d80207150f-data-volume\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:57.094842 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.094811    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8476a109-14a2-44ba-b1b7-69d80207150f-crio-socket\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:57.094896 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.094849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:57.094896 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.094881    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:57.094978 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:57.094952    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 18:25:57.095024 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:57.094982    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 18:25:57.095024 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:57.095019    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm podName:8476a109-14a2-44ba-b1b7-69d80207150f nodeName:}" failed. No retries permitted until 2026-07-12 18:25:57.595001685 +0000 UTC m=+40.478351955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-znnz5" (UID: "8476a109-14a2-44ba-b1b7-69d80207150f") : configmap "kube-rbac-proxy" not found
Jul 12 18:25:57.095103 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:57.095042    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls podName:8476a109-14a2-44ba-b1b7-69d80207150f nodeName:}" failed. No retries permitted until 2026-07-12 18:25:57.595027592 +0000 UTC m=+40.478377848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-znnz5" (UID: "8476a109-14a2-44ba-b1b7-69d80207150f") : secret "insights-runtime-extractor-tls" not found
Jul 12 18:25:57.095143 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.095071    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8476a109-14a2-44ba-b1b7-69d80207150f-crio-socket\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:57.095143 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.095135    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8476a109-14a2-44ba-b1b7-69d80207150f-data-volume\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:57.104328 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.104310    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jkz6\" (UniqueName: \"kubernetes.io/projected/8476a109-14a2-44ba-b1b7-69d80207150f-kube-api-access-4jkz6\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:57.185169 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.185087    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p"]
Jul 12 18:25:57.185331 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.185259    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:57.203622 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.203596    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p"]
Jul 12 18:25:57.203742 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.203684    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p"
Jul 12 18:25:57.206034 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.206008    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-jk8sw\""
Jul 12 18:25:57.209303 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.206492    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 12 18:25:57.209303 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.206684    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 12 18:25:57.262698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.262664    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:57.296594 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.296561    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjdgb\" (UniqueName: \"kubernetes.io/projected/a23e3158-dd56-487c-bb12-3f7c5fec467d-kube-api-access-xjdgb\") pod \"migrator-7f59b54f6c-6bh8p\" (UID: \"a23e3158-dd56-487c-bb12-3f7c5fec467d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p"
Jul 12 18:25:57.358623 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.358593    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:57.397017 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.396984    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjdgb\" (UniqueName: \"kubernetes.io/projected/a23e3158-dd56-487c-bb12-3f7c5fec467d-kube-api-access-xjdgb\") pod \"migrator-7f59b54f6c-6bh8p\" (UID: \"a23e3158-dd56-487c-bb12-3f7c5fec467d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p"
Jul 12 18:25:57.404669 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.404642    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjdgb\" (UniqueName: \"kubernetes.io/projected/a23e3158-dd56-487c-bb12-3f7c5fec467d-kube-api-access-xjdgb\") pod \"migrator-7f59b54f6c-6bh8p\" (UID: \"a23e3158-dd56-487c-bb12-3f7c5fec467d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p"
Jul 12 18:25:57.458979 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.458907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:57.515054 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.515031    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p"
Jul 12 18:25:57.598592 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.598565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:57.598717 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.598607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:57.598770 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:57.598739    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 18:25:57.598848 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:57.598811    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls podName:8476a109-14a2-44ba-b1b7-69d80207150f nodeName:}" failed. No retries permitted until 2026-07-12 18:25:58.598789587 +0000 UTC m=+41.482139845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-znnz5" (UID: "8476a109-14a2-44ba-b1b7-69d80207150f") : secret "insights-runtime-extractor-tls" not found
Jul 12 18:25:57.598848 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:57.598741    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 18:25:57.598963 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:57.598905    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm podName:8476a109-14a2-44ba-b1b7-69d80207150f nodeName:}" failed. No retries permitted until 2026-07-12 18:25:58.598883827 +0000 UTC m=+41.482234079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-znnz5" (UID: "8476a109-14a2-44ba-b1b7-69d80207150f") : configmap "kube-rbac-proxy" not found
Jul 12 18:25:57.661747 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.661707    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p"]
Jul 12 18:25:57.664792 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:25:57.664765    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23e3158_dd56_487c_bb12_3f7c5fec467d.slice/crio-5730f60960a56eb714411989d571ecd56fb4bd58297668c5a416f92eb814b1eb WatchSource:0}: Error finding container 5730f60960a56eb714411989d571ecd56fb4bd58297668c5a416f92eb814b1eb: Status 404 returned error can't find the container with id 5730f60960a56eb714411989d571ecd56fb4bd58297668c5a416f92eb814b1eb
Jul 12 18:25:57.793833 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.793796    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p" event={"ID":"a23e3158-dd56-487c-bb12-3f7c5fec467d","Type":"ContainerStarted","Data":"5730f60960a56eb714411989d571ecd56fb4bd58297668c5a416f92eb814b1eb"}
Jul 12 18:25:57.835092 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.835045    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:57.888057 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:57.888025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:58.103579 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:58.103503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:58.207525 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:58.207493    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:58.208356 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:58.208337    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f4xh5_b098ac18-b58d-4a14-af63-7b1973beb36b/dns-node-resolver/0.log"
Jul 12 18:25:58.304321 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:58.304286    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:58.605694 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:58.605660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:58.605868 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:58.605700    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:25:58.605868 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:58.605778    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 18:25:58.605868 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:58.605824    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm podName:8476a109-14a2-44ba-b1b7-69d80207150f nodeName:}" failed. No retries permitted until 2026-07-12 18:26:00.605811359 +0000 UTC m=+43.489161612 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-znnz5" (UID: "8476a109-14a2-44ba-b1b7-69d80207150f") : configmap "kube-rbac-proxy" not found
Jul 12 18:25:58.606019 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:58.605927    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 18:25:58.606019 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:25:58.605998    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls podName:8476a109-14a2-44ba-b1b7-69d80207150f nodeName:}" failed. No retries permitted until 2026-07-12 18:26:00.605978547 +0000 UTC m=+43.489328816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-znnz5" (UID: "8476a109-14a2-44ba-b1b7-69d80207150f") : secret "insights-runtime-extractor-tls" not found
Jul 12 18:25:59.409720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.409699    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rxs9c_6f73fd2b-9f51-44df-98cb-672c606fc32e/node-ca/0.log"
Jul 12 18:25:59.699739 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.699675    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-db5wh"]
Jul 12 18:25:59.720748 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.720722    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-db5wh"]
Jul 12 18:25:59.720844 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.720810    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:25:59.723104 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.723085    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 12 18:25:59.723529 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.723510    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 12 18:25:59.724026 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.724010    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 12 18:25:59.724026 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.724019    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lfprd\""
Jul 12 18:25:59.724147 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.724031    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 12 18:25:59.739323 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.739292    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:59.799607 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.799576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p" event={"ID":"a23e3158-dd56-487c-bb12-3f7c5fec467d","Type":"ContainerStarted","Data":"2b677b63d69f69e9a4e701644078d40fecd8fc4171835296e970585e2198d66c"}
Jul 12 18:25:59.799710 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.799662    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p" event={"ID":"a23e3158-dd56-487c-bb12-3f7c5fec467d","Type":"ContainerStarted","Data":"8058434e199071da5077c0bcebc5918252f8381c6a98b45202dc6fb4eeeab71a"}
Jul 12 18:25:59.813860 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.813828    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da0b9026-40bb-419f-97e6-da435c4155f0-signing-key\") pod \"service-ca-79f68b9f4f-db5wh\" (UID: \"da0b9026-40bb-419f-97e6-da435c4155f0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:25:59.813975 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.813871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da0b9026-40bb-419f-97e6-da435c4155f0-signing-cabundle\") pod \"service-ca-79f68b9f4f-db5wh\" (UID: \"da0b9026-40bb-419f-97e6-da435c4155f0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:25:59.813975 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.813901    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt2s7\" (UniqueName: \"kubernetes.io/projected/da0b9026-40bb-419f-97e6-da435c4155f0-kube-api-access-nt2s7\") pod \"service-ca-79f68b9f4f-db5wh\" (UID: \"da0b9026-40bb-419f-97e6-da435c4155f0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:25:59.814692 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.814648    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6bh8p" podStartSLOduration=1.087386805 podStartE2EDuration="2.814632449s" totalImagesPullingTime="1.727245644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:59.813650014 +0000 UTC m=+42.697000289" watchObservedRunningTime="2026-07-12 18:25:59.814632449 +0000 UTC m=+42.697982723"
Jul 12 18:25:59.890817 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.890789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:25:59.915143 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.915124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da0b9026-40bb-419f-97e6-da435c4155f0-signing-key\") pod \"service-ca-79f68b9f4f-db5wh\" (UID: \"da0b9026-40bb-419f-97e6-da435c4155f0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:25:59.915236 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.915149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da0b9026-40bb-419f-97e6-da435c4155f0-signing-cabundle\") pod \"service-ca-79f68b9f4f-db5wh\" (UID: \"da0b9026-40bb-419f-97e6-da435c4155f0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:25:59.915236 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.915174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt2s7\" (UniqueName: \"kubernetes.io/projected/da0b9026-40bb-419f-97e6-da435c4155f0-kube-api-access-nt2s7\") pod \"service-ca-79f68b9f4f-db5wh\" (UID: \"da0b9026-40bb-419f-97e6-da435c4155f0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:25:59.915927 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.915908    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da0b9026-40bb-419f-97e6-da435c4155f0-signing-cabundle\") pod \"service-ca-79f68b9f4f-db5wh\" (UID: \"da0b9026-40bb-419f-97e6-da435c4155f0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:25:59.917353 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.917335    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da0b9026-40bb-419f-97e6-da435c4155f0-signing-key\") pod \"service-ca-79f68b9f4f-db5wh\" (UID: \"da0b9026-40bb-419f-97e6-da435c4155f0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:25:59.923445 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.923423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt2s7\" (UniqueName: \"kubernetes.io/projected/da0b9026-40bb-419f-97e6-da435c4155f0-kube-api-access-nt2s7\") pod \"service-ca-79f68b9f4f-db5wh\" (UID: \"da0b9026-40bb-419f-97e6-da435c4155f0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:25:59.990672 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:25:59.990642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:00.008777 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:00.008751    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7msr2_2be4bca4-2772-4c13-a24b-eebc9d61cc7d/serve-healthcheck-canary/0.log"
Jul 12 18:26:00.033119 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:00.033088    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh"
Jul 12 18:26:00.165357 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:00.165328    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-db5wh"]
Jul 12 18:26:00.167901 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:26:00.167881    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0b9026_40bb_419f_97e6_da435c4155f0.slice/crio-db679d6ee3ee6557d04388e86ebd4d532e0b9293192e0a09bc9012abddf33187 WatchSource:0}: Error finding container db679d6ee3ee6557d04388e86ebd4d532e0b9293192e0a09bc9012abddf33187: Status 404 returned error can't find the container with id db679d6ee3ee6557d04388e86ebd4d532e0b9293192e0a09bc9012abddf33187
Jul 12 18:26:00.318015 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:00.317938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:26:00.318015 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:00.317973    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:26:00.318241 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:00.318092    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 18:26:00.318241 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:00.318107    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 18:26:00.318241 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:00.318121    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcbcb4984-kjk44: secret "image-registry-tls" not found
Jul 12 18:26:00.318241 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:00.318155    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls podName:cee6a773-686d-4ef4-a904-110a2826bce6 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:08.318139777 +0000 UTC m=+51.201490029 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls") pod "dns-default-h8jls" (UID: "cee6a773-686d-4ef4-a904-110a2826bce6") : secret "dns-default-metrics-tls" not found
Jul 12 18:26:00.318241 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:00.318171    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls podName:96c3b950-1121-433a-9eff-8dfa9a0034a0 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:08.31816345 +0000 UTC m=+51.201513702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls") pod "image-registry-6dcbcb4984-kjk44" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0") : secret "image-registry-tls" not found
Jul 12 18:26:00.620209 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:00.620126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:26:00.620209 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:00.620179    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:26:00.620705 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:00.620306    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 18:26:00.620705 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:00.620379    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls podName:8476a109-14a2-44ba-b1b7-69d80207150f nodeName:}" failed. No retries permitted until 2026-07-12 18:26:04.620357483 +0000 UTC m=+47.503707737 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-znnz5" (UID: "8476a109-14a2-44ba-b1b7-69d80207150f") : secret "insights-runtime-extractor-tls" not found
Jul 12 18:26:00.620705 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:00.620314    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 18:26:00.620705 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:00.620479    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm podName:8476a109-14a2-44ba-b1b7-69d80207150f nodeName:}" failed. No retries permitted until 2026-07-12 18:26:04.620443693 +0000 UTC m=+47.503793957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-znnz5" (UID: "8476a109-14a2-44ba-b1b7-69d80207150f") : configmap "kube-rbac-proxy" not found
Jul 12 18:26:00.712054 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:00.712016    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:00.803719 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:00.803684    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh" event={"ID":"da0b9026-40bb-419f-97e6-da435c4155f0","Type":"ContainerStarted","Data":"db679d6ee3ee6557d04388e86ebd4d532e0b9293192e0a09bc9012abddf33187"}
Jul 12 18:26:01.128813 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:01.128782    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:02.810011 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:02.809978    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh" event={"ID":"da0b9026-40bb-419f-97e6-da435c4155f0","Type":"ContainerStarted","Data":"6943a0d2bb470dec83bde2f33e0dbefec9fe79e4bf4151b9a3a2442c78c1618b"}
Jul 12 18:26:02.824401 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:02.824352    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-db5wh" podStartSLOduration=1.5417257370000002 podStartE2EDuration="3.824336177s" totalImagesPullingTime="2.28261044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:02.823542328 +0000 UTC m=+45.706892603" watchObservedRunningTime="2026-07-12 18:26:02.824336177 +0000 UTC m=+45.707686501"
Jul 12 18:26:03.377379 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:03.377342    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:03.558513 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:03.558475    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:03.795188 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:03.795152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:03.896887 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:03.896814    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:04.021802 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:04.021764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:04.205851 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:04.205750    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:04.309542 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:04.309491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:04.469991 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:04.469898    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:04.657109 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:04.657074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:26:04.657283 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:04.657117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:26:04.657283 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:04.657219    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 18:26:04.657283 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:04.657279    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls podName:8476a109-14a2-44ba-b1b7-69d80207150f nodeName:}" failed. No retries permitted until 2026-07-12 18:26:12.657263866 +0000 UTC m=+55.540614118 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-znnz5" (UID: "8476a109-14a2-44ba-b1b7-69d80207150f") : secret "insights-runtime-extractor-tls" not found
Jul 12 18:26:04.657738 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:04.657715    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8476a109-14a2-44ba-b1b7-69d80207150f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:26:05.269281 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:05.269245    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:05.864060 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:05.864019    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:06.068145 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:06.068108    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:06.268591 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:06.268553    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:06.468220 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:06.468192    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:07.061856 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:07.061822    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:07.184438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:07.184405    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:07.285739 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:07.285706    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:07.386341 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:07.386258    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:07.487268 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:07.487238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:07.666446 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:07.666364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:08.062513 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.062477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:08.162895 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.162861    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:08.265005 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.264971    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:08.390704 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.390604    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:26:08.390704 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.390647    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:26:08.393146 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.393108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls\") pod \"image-registry-6dcbcb4984-kjk44\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") " pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:26:08.393260 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.393125    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee6a773-686d-4ef4-a904-110a2826bce6-metrics-tls\") pod \"dns-default-h8jls\" (UID: \"cee6a773-686d-4ef4-a904-110a2826bce6\") " pod="openshift-dns/dns-default-h8jls"
Jul 12 18:26:08.467443 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.467402    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:26:08.479664 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.479627    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h8jls"
Jul 12 18:26:08.618632 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.618599    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dcbcb4984-kjk44"]
Jul 12 18:26:08.621633 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:26:08.621604    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c3b950_1121_433a_9eff_8dfa9a0034a0.slice/crio-3c20e62445547b851488bd5d5e984de3713d75f0e061fcdd5327e286e3dbb36d WatchSource:0}: Error finding container 3c20e62445547b851488bd5d5e984de3713d75f0e061fcdd5327e286e3dbb36d: Status 404 returned error can't find the container with id 3c20e62445547b851488bd5d5e984de3713d75f0e061fcdd5327e286e3dbb36d
Jul 12 18:26:08.635439 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.635416    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h8jls"]
Jul 12 18:26:08.638581 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:26:08.638561    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee6a773_686d_4ef4_a904_110a2826bce6.slice/crio-be966537afc8586131db91e2198b870760cdca13f8363862edbae898991c5db1 WatchSource:0}: Error finding container be966537afc8586131db91e2198b870760cdca13f8363862edbae898991c5db1: Status 404 returned error can't find the container with id be966537afc8586131db91e2198b870760cdca13f8363862edbae898991c5db1
Jul 12 18:26:08.665800 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.665772    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:08.831866 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.831827    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44" event={"ID":"96c3b950-1121-433a-9eff-8dfa9a0034a0","Type":"ContainerStarted","Data":"ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52"}
Jul 12 18:26:08.832035 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.831916    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:26:08.832035 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.831941    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44" event={"ID":"96c3b950-1121-433a-9eff-8dfa9a0034a0","Type":"ContainerStarted","Data":"3c20e62445547b851488bd5d5e984de3713d75f0e061fcdd5327e286e3dbb36d"}
Jul 12 18:26:08.833067 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.833042    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h8jls" event={"ID":"cee6a773-686d-4ef4-a904-110a2826bce6","Type":"ContainerStarted","Data":"be966537afc8586131db91e2198b870760cdca13f8363862edbae898991c5db1"}
Jul 12 18:26:08.851717 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:08.851669    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44" podStartSLOduration=20.851654472 podStartE2EDuration="20.851654472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:08.850646706 +0000 UTC m=+51.733996980" watchObservedRunningTime="2026-07-12 18:26:08.851654472 +0000 UTC m=+51.735004724"
Jul 12 18:26:09.466887 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:09.466854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:09.657169 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:09.657132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:09.760294 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:09.760266    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:10.063575 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:10.063465    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:10.269546 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:10.269510    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:10.666211 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:10.666170    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:11.469085 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:11.469045    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:11.844261 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:11.844232    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h8jls" event={"ID":"cee6a773-686d-4ef4-a904-110a2826bce6","Type":"ContainerStarted","Data":"70da2dc4753a659fde986d41b38be852ebb2e9d7fa244befb46d05410bdb0ed5"}
Jul 12 18:26:11.844677 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:11.844281    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h8jls" event={"ID":"cee6a773-686d-4ef4-a904-110a2826bce6","Type":"ContainerStarted","Data":"737c5d7140ab289ff9d48a0c690ef63e74692217037bed15786864442367885d"}
Jul 12 18:26:11.844677 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:11.844345    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h8jls"
Jul 12 18:26:11.861790 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:11.861734    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-h8jls" podStartSLOduration=17.510579614 podStartE2EDuration="19.861716792s" totalImagesPullingTime="2.351137178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:11.860485984 +0000 UTC m=+54.743836256" watchObservedRunningTime="2026-07-12 18:26:11.861716792 +0000 UTC m=+54.745067067"
Jul 12 18:26:11.867027 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:11.866996    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:12.067003 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:12.066967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:12.266874 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:12.266843    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:12.463196 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:12.463166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:12.728505 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:12.728479    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:26:12.730738 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:12.730713    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8476a109-14a2-44ba-b1b7-69d80207150f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnz5\" (UID: \"8476a109-14a2-44ba-b1b7-69d80207150f\") " pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:26:12.794619 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:12.794584    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-znnz5"
Jul 12 18:26:12.915109 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:12.915080    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-znnz5"]
Jul 12 18:26:12.918285 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:26:12.918263    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8476a109_14a2_44ba_b1b7_69d80207150f.slice/crio-634036e782072e415182d6192e38a43aa10fb2e173855b798e615c2832e0c3bd WatchSource:0}: Error finding container 634036e782072e415182d6192e38a43aa10fb2e173855b798e615c2832e0c3bd: Status 404 returned error can't find the container with id 634036e782072e415182d6192e38a43aa10fb2e173855b798e615c2832e0c3bd
Jul 12 18:26:13.120682 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:13.120608    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:13.221344 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:13.221313    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:13.321999 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:13.321967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:13.435366 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:13.435282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:13.529655 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:13.529570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:13.634233 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:13.634159    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:13.731349 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:13.731315    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:13.832144 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:13.832073    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:13.855083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:13.855049    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-znnz5" event={"ID":"8476a109-14a2-44ba-b1b7-69d80207150f","Type":"ContainerStarted","Data":"634036e782072e415182d6192e38a43aa10fb2e173855b798e615c2832e0c3bd"}
Jul 12 18:26:13.933108 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:13.933006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:14.067507 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:14.067414    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:14.234633 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:14.234515    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:14.296229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:14.296197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:14.632784 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:14.632754    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:14.734299 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:14.734268    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:15.012034 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:15.012002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:15.212064 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:15.212036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:15.517923 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:15.517880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:16.032525 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:16.032491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:16.099955 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:16.099925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:16.867668 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:16.867636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-znnz5" event={"ID":"8476a109-14a2-44ba-b1b7-69d80207150f","Type":"ContainerStarted","Data":"a2893098d63b26cc95b082397759450d4f4a78bc1fd88af7b2d7703d3c7f1bb3"}
Jul 12 18:26:17.763982 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:17.763955    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p8p68"
Jul 12 18:26:17.872789 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:17.872759    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-znnz5" event={"ID":"8476a109-14a2-44ba-b1b7-69d80207150f","Type":"ContainerStarted","Data":"1064e25cbeeff342b3965f99782ab68558859a6f31a8ae0df8c6246b5031e516"}
Jul 12 18:26:18.925642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.925609    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"]
Jul 12 18:26:18.945129 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.945091    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"]
Jul 12 18:26:18.945295 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.945223    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"
Jul 12 18:26:18.948184 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.948130    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-nrttc\""
Jul 12 18:26:18.948184 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.948184    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 12 18:26:18.949182 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.949162    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 12 18:26:18.949428 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.949409    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 12 18:26:18.949538 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.949409    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 12 18:26:18.965546 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.965479    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"]
Jul 12 18:26:18.965546 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.965518    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"]
Jul 12 18:26:18.965705 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.965569    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:18.966229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.966203    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dcbcb4984-kjk44"]
Jul 12 18:26:18.968583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.968561    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 12 18:26:18.968809 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.968791    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 12 18:26:18.969034 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.969018    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 12 18:26:18.969283 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.969264    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 12 18:26:18.982178 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.982158    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jpd9\" (UniqueName: \"kubernetes.io/projected/3c8f177d-38de-4fbc-9eba-0be49bd4891e-kube-api-access-8jpd9\") pod \"managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz\" (UID: \"3c8f177d-38de-4fbc-9eba-0be49bd4891e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"
Jul 12 18:26:18.982280 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:18.982210    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c8f177d-38de-4fbc-9eba-0be49bd4891e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz\" (UID: \"3c8f177d-38de-4fbc-9eba-0be49bd4891e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"
Jul 12 18:26:19.036192 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.036154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:19.082625 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.082585    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.082625 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.082630    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b792cb83-faab-4f54-b3fd-a76411344b3b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.082827 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.082695    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-ca\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.082827 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.082734    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6zd7\" (UniqueName: \"kubernetes.io/projected/b792cb83-faab-4f54-b3fd-a76411344b3b-kube-api-access-w6zd7\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.082827 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.082759    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-hub\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.082989 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.082827    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.082989 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.082898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jpd9\" (UniqueName: \"kubernetes.io/projected/3c8f177d-38de-4fbc-9eba-0be49bd4891e-kube-api-access-8jpd9\") pod \"managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz\" (UID: \"3c8f177d-38de-4fbc-9eba-0be49bd4891e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"
Jul 12 18:26:19.082989 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.082958    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c8f177d-38de-4fbc-9eba-0be49bd4891e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz\" (UID: \"3c8f177d-38de-4fbc-9eba-0be49bd4891e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"
Jul 12 18:26:19.085619 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.085591    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c8f177d-38de-4fbc-9eba-0be49bd4891e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz\" (UID: \"3c8f177d-38de-4fbc-9eba-0be49bd4891e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"
Jul 12 18:26:19.091969 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.091949    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-677fddf46f-2hds5"]
Jul 12 18:26:19.094238 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.094206    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jpd9\" (UniqueName: \"kubernetes.io/projected/3c8f177d-38de-4fbc-9eba-0be49bd4891e-kube-api-access-8jpd9\") pod \"managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz\" (UID: \"3c8f177d-38de-4fbc-9eba-0be49bd4891e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"
Jul 12 18:26:19.117583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.117560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:19.118110 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.118086    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-677fddf46f-2hds5"]
Jul 12 18:26:19.118228 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.118171    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.184038 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.183956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.184038 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b5204ff-2f07-435b-a169-2210ca8a2b6b-registry-certificates\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.184229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b792cb83-faab-4f54-b3fd-a76411344b3b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.184229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184123    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b5204ff-2f07-435b-a169-2210ca8a2b6b-ca-trust-extracted\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.184229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184146    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b5204ff-2f07-435b-a169-2210ca8a2b6b-installation-pull-secrets\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.184229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184179    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-ca\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.184229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6zd7\" (UniqueName: \"kubernetes.io/projected/b792cb83-faab-4f54-b3fd-a76411344b3b-kube-api-access-w6zd7\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.184229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184229    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-hub\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.184571 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b5204ff-2f07-435b-a169-2210ca8a2b6b-bound-sa-token\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.184571 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184306    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.184571 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b5204ff-2f07-435b-a169-2210ca8a2b6b-trusted-ca\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.184709 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184614    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjtcp\" (UniqueName: \"kubernetes.io/projected/6b5204ff-2f07-435b-a169-2210ca8a2b6b-kube-api-access-vjtcp\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.184709 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184667    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b5204ff-2f07-435b-a169-2210ca8a2b6b-image-registry-private-configuration\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.184709 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.184692    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5204ff-2f07-435b-a169-2210ca8a2b6b-registry-tls\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.185362 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.185333    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b792cb83-faab-4f54-b3fd-a76411344b3b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.186938 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.186900    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.187061 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.187037    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:19.187615 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.187597    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-hub\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.187728 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.187625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-ca\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.187790 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.187734    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b792cb83-faab-4f54-b3fd-a76411344b3b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.193172 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.193151    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6zd7\" (UniqueName: \"kubernetes.io/projected/b792cb83-faab-4f54-b3fd-a76411344b3b-kube-api-access-w6zd7\") pod \"cluster-proxy-proxy-agent-5c9865c75b-wgx9n\" (UID: \"b792cb83-faab-4f54-b3fd-a76411344b3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.264880 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.264852    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"
Jul 12 18:26:19.277515 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.277483    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"
Jul 12 18:26:19.285470 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.285431    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b5204ff-2f07-435b-a169-2210ca8a2b6b-registry-certificates\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.285599 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.285481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b5204ff-2f07-435b-a169-2210ca8a2b6b-ca-trust-extracted\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.285599 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.285512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b5204ff-2f07-435b-a169-2210ca8a2b6b-installation-pull-secrets\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.285599 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.285555    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b5204ff-2f07-435b-a169-2210ca8a2b6b-bound-sa-token\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.285765 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.285611    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b5204ff-2f07-435b-a169-2210ca8a2b6b-trusted-ca\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.285765 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.285666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjtcp\" (UniqueName: \"kubernetes.io/projected/6b5204ff-2f07-435b-a169-2210ca8a2b6b-kube-api-access-vjtcp\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.285765 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.285706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b5204ff-2f07-435b-a169-2210ca8a2b6b-image-registry-private-configuration\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.285765 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.285735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5204ff-2f07-435b-a169-2210ca8a2b6b-registry-tls\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.285981 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.285889    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b5204ff-2f07-435b-a169-2210ca8a2b6b-ca-trust-extracted\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.286387 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.286332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b5204ff-2f07-435b-a169-2210ca8a2b6b-registry-certificates\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.286804 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.286771    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b5204ff-2f07-435b-a169-2210ca8a2b6b-trusted-ca\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.287917 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.287863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:19.288697 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.288677    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5204ff-2f07-435b-a169-2210ca8a2b6b-registry-tls\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.288783 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.288746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b5204ff-2f07-435b-a169-2210ca8a2b6b-image-registry-private-configuration\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.288837 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.288824    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b5204ff-2f07-435b-a169-2210ca8a2b6b-installation-pull-secrets\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.294281 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.294024    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b5204ff-2f07-435b-a169-2210ca8a2b6b-bound-sa-token\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.294281 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.294237    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjtcp\" (UniqueName: \"kubernetes.io/projected/6b5204ff-2f07-435b-a169-2210ca8a2b6b-kube-api-access-vjtcp\") pod \"image-registry-677fddf46f-2hds5\" (UID: \"6b5204ff-2f07-435b-a169-2210ca8a2b6b\") " pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.434573 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.434490    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:19.750267 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.750237    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:19.813312 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:19.813275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:20.123477 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.123420    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-677fddf46f-2hds5"]
Jul 12 18:26:20.128054 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:26:20.128014    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5204ff_2f07_435b_a169_2210ca8a2b6b.slice/crio-0bd5098ee0857ac139143640713c63ea529bcf2f424827d74a9b40fa1da74ccf WatchSource:0}: Error finding container 0bd5098ee0857ac139143640713c63ea529bcf2f424827d74a9b40fa1da74ccf: Status 404 returned error can't find the container with id 0bd5098ee0857ac139143640713c63ea529bcf2f424827d74a9b40fa1da74ccf
Jul 12 18:26:20.141868 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.141839    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n"]
Jul 12 18:26:20.146889 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:26:20.146859    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb792cb83_faab_4f54_b3fd_a76411344b3b.slice/crio-fc7e36654a529647a9e44e71f6a4ca30d1e4bb0d615089c3cb9c5eb4e60a8b53 WatchSource:0}: Error finding container fc7e36654a529647a9e44e71f6a4ca30d1e4bb0d615089c3cb9c5eb4e60a8b53: Status 404 returned error can't find the container with id fc7e36654a529647a9e44e71f6a4ca30d1e4bb0d615089c3cb9c5eb4e60a8b53
Jul 12 18:26:20.152341 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.152311    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz"]
Jul 12 18:26:20.164373 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:26:20.164349    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c8f177d_38de_4fbc_9eba_0be49bd4891e.slice/crio-6a3ca46fc33373dbe54a0cedcb3d42dbe3cd8faf22f8a07f628e4dce75e7daca WatchSource:0}: Error finding container 6a3ca46fc33373dbe54a0cedcb3d42dbe3cd8faf22f8a07f628e4dce75e7daca: Status 404 returned error can't find the container with id 6a3ca46fc33373dbe54a0cedcb3d42dbe3cd8faf22f8a07f628e4dce75e7daca
Jul 12 18:26:20.826404 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.826294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:20.877656 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.877626    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:20.885233 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.885203    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-znnz5" event={"ID":"8476a109-14a2-44ba-b1b7-69d80207150f","Type":"ContainerStarted","Data":"2ecec1bbe3a66a187f761aef820ef03618c13d9bac409142554097b459a3d967"}
Jul 12 18:26:20.886720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.886658    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n" event={"ID":"b792cb83-faab-4f54-b3fd-a76411344b3b","Type":"ContainerStarted","Data":"fc7e36654a529647a9e44e71f6a4ca30d1e4bb0d615089c3cb9c5eb4e60a8b53"}
Jul 12 18:26:20.888158 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.888120    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz" event={"ID":"3c8f177d-38de-4fbc-9eba-0be49bd4891e","Type":"ContainerStarted","Data":"6a3ca46fc33373dbe54a0cedcb3d42dbe3cd8faf22f8a07f628e4dce75e7daca"}
Jul 12 18:26:20.890255 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.890230    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-677fddf46f-2hds5" event={"ID":"6b5204ff-2f07-435b-a169-2210ca8a2b6b","Type":"ContainerStarted","Data":"25a4642e67e0525a0aaf4d788ddd463fe8ee6319b3f14fda32e343b1585f2e6f"}
Jul 12 18:26:20.890367 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.890277    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-677fddf46f-2hds5" event={"ID":"6b5204ff-2f07-435b-a169-2210ca8a2b6b","Type":"ContainerStarted","Data":"0bd5098ee0857ac139143640713c63ea529bcf2f424827d74a9b40fa1da74ccf"}
Jul 12 18:26:20.890422 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.890391    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:20.903504 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.903444    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-znnz5" podStartSLOduration=17.924606684 podStartE2EDuration="24.903428623s" totalImagesPullingTime="6.978821939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:20.901742481 +0000 UTC m=+63.785092757" watchObservedRunningTime="2026-07-12 18:26:20.903428623 +0000 UTC m=+63.786778903"
Jul 12 18:26:20.932968 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:20.932915    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-677fddf46f-2hds5" podStartSLOduration=1.9328957409999998 podStartE2EDuration="1.932895741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:26:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:20.931702763 +0000 UTC m=+63.815053036" watchObservedRunningTime="2026-07-12 18:26:20.932895741 +0000 UTC m=+63.816246017"
Jul 12 18:26:21.852295 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:21.851577    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h8jls"
Jul 12 18:26:22.292518 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:22.292486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:22.346377 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:22.346333    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:23.326012 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.325980    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:26:23.326484 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.326045    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:26:23.326484 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.326094    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmscs\" (UniqueName: \"kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs\") pod \"network-check-target-q77wh\" (UID: \"d11c8121-7766-4973-9398-0ed4c73fd2e3\") " pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:26:23.328337 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.328315    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 12 18:26:23.329078 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.329061    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 12 18:26:23.329078 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.329069    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 12 18:26:23.339204 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.339181    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 12 18:26:23.339507 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.339486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0-metrics-certs\") pod \"network-metrics-daemon-cbzdr\" (UID: \"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0\") " pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:26:23.339610 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.339554    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adc887e4-8e0a-4ed3-9978-792e0a1573bb-original-pull-secret\") pod \"global-pull-secret-syncer-4kw4m\" (UID: \"adc887e4-8e0a-4ed3-9978-792e0a1573bb\") " pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:26:23.350192 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.350169    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmscs\" (UniqueName: \"kubernetes.io/projected/d11c8121-7766-4973-9398-0ed4c73fd2e3-kube-api-access-zmscs\") pod \"network-check-target-q77wh\" (UID: \"d11c8121-7766-4973-9398-0ed4c73fd2e3\") " pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:26:23.511066 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.511032    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p8prd\""
Jul 12 18:26:23.515899 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.515875    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4kw4m"
Jul 12 18:26:23.518860 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.518840    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:26:23.523627 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.523602    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gx6mz\""
Jul 12 18:26:23.532451 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:23.532426    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbzdr"
Jul 12 18:26:24.138767 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.138588    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4kw4m"]
Jul 12 18:26:24.152653 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.152623    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cbzdr"]
Jul 12 18:26:24.155344 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:26:24.155309    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a776d1_4aa1_4bdc_86f5_b8bd2097e5e0.slice/crio-4ec71f8bf73f2c90907dcb3f022a0664c5eb1efb5f9867f7db09b647a5d7a418 WatchSource:0}: Error finding container 4ec71f8bf73f2c90907dcb3f022a0664c5eb1efb5f9867f7db09b647a5d7a418: Status 404 returned error can't find the container with id 4ec71f8bf73f2c90907dcb3f022a0664c5eb1efb5f9867f7db09b647a5d7a418
Jul 12 18:26:24.172508 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.172484    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q77wh"]
Jul 12 18:26:24.176062 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:26:24.176037    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11c8121_7766_4973_9398_0ed4c73fd2e3.slice/crio-500908a8a5f5847afe9c13927288ae4a7e1b669eced78966f1b15e3da5583af2 WatchSource:0}: Error finding container 500908a8a5f5847afe9c13927288ae4a7e1b669eced78966f1b15e3da5583af2: Status 404 returned error can't find the container with id 500908a8a5f5847afe9c13927288ae4a7e1b669eced78966f1b15e3da5583af2
Jul 12 18:26:24.910917 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.910878    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n" event={"ID":"b792cb83-faab-4f54-b3fd-a76411344b3b","Type":"ContainerStarted","Data":"9e4c2db5b85c35d44118650b6563bad696c1ae1c55f04f00b4e2a8765118ab9b"}
Jul 12 18:26:24.911383 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.910929    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n" event={"ID":"b792cb83-faab-4f54-b3fd-a76411344b3b","Type":"ContainerStarted","Data":"2f39b7eada0774d34327c62787864edff961a2b7fddffdd58b5a0bb6fe405973"}
Jul 12 18:26:24.911383 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.910949    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n" event={"ID":"b792cb83-faab-4f54-b3fd-a76411344b3b","Type":"ContainerStarted","Data":"0a556618d0d962761f7eef78c9c55c266405cee398de3185bb1e0ff443c38f80"}
Jul 12 18:26:24.912356 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.912333    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q77wh" event={"ID":"d11c8121-7766-4973-9398-0ed4c73fd2e3","Type":"ContainerStarted","Data":"500908a8a5f5847afe9c13927288ae4a7e1b669eced78966f1b15e3da5583af2"}
Jul 12 18:26:24.914411 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.913979    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz" event={"ID":"3c8f177d-38de-4fbc-9eba-0be49bd4891e","Type":"ContainerStarted","Data":"0b91bd2222aca05bb2702b63046bc637c8c0ad8f8d0e90553764ce710713e11a"}
Jul 12 18:26:24.915746 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.915725    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cbzdr" event={"ID":"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0","Type":"ContainerStarted","Data":"4ec71f8bf73f2c90907dcb3f022a0664c5eb1efb5f9867f7db09b647a5d7a418"}
Jul 12 18:26:24.917572 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.917552    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4kw4m" event={"ID":"adc887e4-8e0a-4ed3-9978-792e0a1573bb","Type":"ContainerStarted","Data":"ae203d0b2b4b4158343db1704fc5cde75445df9afb16ab53068800b19603993e"}
Jul 12 18:26:24.932250 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.932194    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9865c75b-wgx9n" podStartSLOduration=3.095731838 podStartE2EDuration="6.932174792s" totalImagesPullingTime="3.836442954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:24.929942023 +0000 UTC m=+67.813292302" watchObservedRunningTime="2026-07-12 18:26:24.932174792 +0000 UTC m=+67.815525067"
Jul 12 18:26:24.945858 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:24.945243    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd856d4dd-7vjmz" podStartSLOduration=3.143629016 podStartE2EDuration="6.945224264s" totalImagesPullingTime="3.801595248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:24.944099899 +0000 UTC m=+67.827450176" watchObservedRunningTime="2026-07-12 18:26:24.945224264 +0000 UTC m=+67.828574539"
Jul 12 18:26:25.923761 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:25.923652    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cbzdr" event={"ID":"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0","Type":"ContainerStarted","Data":"cef90390c5fdb27e4046678e56506915ee355d83972880a3c8ef5da79e2d7476"}
Jul 12 18:26:25.923761 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:25.923722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cbzdr" event={"ID":"77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0","Type":"ContainerStarted","Data":"7f8ca6ceed6819094eda85af596d291fd4dc3df89d403eead6033310d15bd22f"}
Jul 12 18:26:25.939960 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:25.939772    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cbzdr" podStartSLOduration=67.9381949 podStartE2EDuration="1m8.939757075s" totalImagesPullingTime="1.001562175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:25.939490053 +0000 UTC m=+68.822840330" watchObservedRunningTime="2026-07-12 18:26:25.939757075 +0000 UTC m=+68.823107350"
Jul 12 18:26:26.058705 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:26.058657    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:26.158683 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:26.158643    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:27.432807 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:27.432758    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:27.488286 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:27.488256    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:28.937764 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:28.937701    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4kw4m" event={"ID":"adc887e4-8e0a-4ed3-9978-792e0a1573bb","Type":"ContainerStarted","Data":"8501434c30bc5f0330b23e627285a5addd98d81a2a78f72c5c57d341e10d4338"}
Jul 12 18:26:28.940369 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:28.940336    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q77wh" event={"ID":"d11c8121-7766-4973-9398-0ed4c73fd2e3","Type":"ContainerStarted","Data":"52e4b6ed3f9942826dbb1d1af44d891e16778e2590f1657ecb5b5368d8532c7a"}
Jul 12 18:26:28.940834 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:28.940812    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:26:28.955749 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:28.955706    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4kw4m" podStartSLOduration=67.38485293 podStartE2EDuration="1m11.955690609s" totalImagesPullingTime="4.570837679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:28.954693154 +0000 UTC m=+71.838043431" watchObservedRunningTime="2026-07-12 18:26:28.955690609 +0000 UTC m=+71.839040883"
Jul 12 18:26:28.969108 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:28.969015    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q77wh" podStartSLOduration=67.434217109 podStartE2EDuration="1m11.969001744s" totalImagesPullingTime="4.534784635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:28.968668247 +0000 UTC m=+71.852018521" watchObservedRunningTime="2026-07-12 18:26:28.969001744 +0000 UTC m=+71.852352019"
Jul 12 18:26:28.971769 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:28.971744    2562 patch_prober.go:28] interesting pod/image-registry-6dcbcb4984-kjk44 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 12 18:26:28.971859 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:28.971786    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44" podUID="96c3b950-1121-433a-9eff-8dfa9a0034a0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 12 18:26:29.048307 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:29.048277    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:29.117335 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:29.117309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:29.699323 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:29.699291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:29.788582 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:29.788549    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:29.882981 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:29.882947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:29.984753 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:29.984728    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:30.085490 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:30.085442    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:30.188218 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:30.188190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:30.810572 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:30.810543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:31.436370 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:31.436339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:31.488330 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:31.488295    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:32.245983 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:32.245952    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:32.303322 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:32.303289    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:32.356955 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:32.356922    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:32.777818 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:32.777789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:32.849805 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:32.849774    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:33.235509 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:33.235481    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:33.285903 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:33.285876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:34.868276 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:34.868238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:34.936744 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:34.936709    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:35.289866 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:35.289835    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:36.255112 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:36.255080    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:36.355602 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:36.355565    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:36.653326 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:36.653240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:36.749016 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:36.748988    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:37.107416 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:37.107382    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:37.557878 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:37.557844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:38.971445 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:38.971410    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:26:38.971861 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:38.971835    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:39.072607 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:39.072503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:40.016092 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.016057    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:40.109306 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.109268    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:40.235195 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.235160    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:40.300264 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.300186    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f56gm"]
Jul 12 18:26:40.303859 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.303835    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.306865 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.306838    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 12 18:26:40.307012 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.307000    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 12 18:26:40.307139 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.307118    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 12 18:26:40.307259 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.307193    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wnhfx\""
Jul 12 18:26:40.307700 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.307544    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 12 18:26:40.307700 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.307572    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 12 18:26:40.307841 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.307759    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 12 18:26:40.334193 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.334168    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:40.431587 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.431551    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:40.471267 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.471235    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-wtmp\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.471449 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.471291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-textfile\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.471449 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.471309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.471449 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.471423    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.471627 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.471501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a28fc4fa-db9a-4279-aa00-fedb855f66a3-root\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.471627 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.471532    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a28fc4fa-db9a-4279-aa00-fedb855f66a3-metrics-client-ca\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.471627 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.471556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w69s\" (UniqueName: \"kubernetes.io/projected/a28fc4fa-db9a-4279-aa00-fedb855f66a3-kube-api-access-8w69s\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.471627 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.471608    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-tls\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.471815 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.471629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a28fc4fa-db9a-4279-aa00-fedb855f66a3-sys\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.532374 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.532334    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:40.572563 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-textfile\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.572563 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.572563 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.572808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572588    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a28fc4fa-db9a-4279-aa00-fedb855f66a3-root\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.572808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572617    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a28fc4fa-db9a-4279-aa00-fedb855f66a3-metrics-client-ca\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.572808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572637    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w69s\" (UniqueName: \"kubernetes.io/projected/a28fc4fa-db9a-4279-aa00-fedb855f66a3-kube-api-access-8w69s\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.572808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572674    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-tls\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.572808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a28fc4fa-db9a-4279-aa00-fedb855f66a3-sys\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.572808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-wtmp\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.573088 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-textfile\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.573088 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572905    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a28fc4fa-db9a-4279-aa00-fedb855f66a3-sys\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.573088 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.572905    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-wtmp\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.573252 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.573226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.573412 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.573320    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a28fc4fa-db9a-4279-aa00-fedb855f66a3-root\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.573544 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:40.573423    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 12 18:26:40.573643 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.573423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a28fc4fa-db9a-4279-aa00-fedb855f66a3-metrics-client-ca\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.573643 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:40.573601    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-tls podName:a28fc4fa-db9a-4279-aa00-fedb855f66a3 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:41.073581016 +0000 UTC m=+83.956931281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-tls") pod "node-exporter-f56gm" (UID: "a28fc4fa-db9a-4279-aa00-fedb855f66a3") : secret "node-exporter-tls" not found
Jul 12 18:26:40.575297 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.575277    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.582337 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.582307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w69s\" (UniqueName: \"kubernetes.io/projected/a28fc4fa-db9a-4279-aa00-fedb855f66a3-kube-api-access-8w69s\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:40.627999 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.627971    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:40.729383 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:40.729300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:41.072117 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.072079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:41.077184 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.077165    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-tls\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:41.079354 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.079332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a28fc4fa-db9a-4279-aa00-fedb855f66a3-node-exporter-tls\") pod \"node-exporter-f56gm\" (UID: \"a28fc4fa-db9a-4279-aa00-fedb855f66a3\") " pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:41.214647 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.214615    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f56gm"
Jul 12 18:26:41.225615 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:26:41.225584    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28fc4fa_db9a_4279_aa00_fedb855f66a3.slice/crio-24ece5b74ee1e4097dafb7264e6d9543c09d53172c690d99009e1d17f3001e39 WatchSource:0}: Error finding container 24ece5b74ee1e4097dafb7264e6d9543c09d53172c690d99009e1d17f3001e39: Status 404 returned error can't find the container with id 24ece5b74ee1e4097dafb7264e6d9543c09d53172c690d99009e1d17f3001e39
Jul 12 18:26:41.246991 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.246962    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:41.336820 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.336734    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:41.451367 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.451337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:41.674408 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.674328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:41.777093 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.777061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:41.875257 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.875225    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:41.901566 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.901536    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-677fddf46f-2hds5"
Jul 12 18:26:41.958832 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.958748    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:41.987824 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:41.987789    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f56gm" event={"ID":"a28fc4fa-db9a-4279-aa00-fedb855f66a3","Type":"ContainerStarted","Data":"24ece5b74ee1e4097dafb7264e6d9543c09d53172c690d99009e1d17f3001e39"}
Jul 12 18:26:42.038538 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:42.038495    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:42.138577 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:42.138546    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:42.569452 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:42.569418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:42.669972 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:42.669941    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:42.996377 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:42.996347    2562 generic.go:356] "Generic (PLEG): container finished" podID="a28fc4fa-db9a-4279-aa00-fedb855f66a3" containerID="c1486446afe2a4c7ac32603c84f0c9ce76b4a85956fa05b4f38a12d699844745" exitCode=0
Jul 12 18:26:42.996553 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:42.996430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f56gm" event={"ID":"a28fc4fa-db9a-4279-aa00-fedb855f66a3","Type":"ContainerDied","Data":"c1486446afe2a4c7ac32603c84f0c9ce76b4a85956fa05b4f38a12d699844745"}
Jul 12 18:26:43.246211 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:43.246177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:43.346689 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:43.346605    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:43.407668 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:43.407637    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:43.586733 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:43.586697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:43.997042 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:43.997007    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44" podUID="96c3b950-1121-433a-9eff-8dfa9a0034a0" containerName="registry" containerID="cri-o://ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52" gracePeriod=30
Jul 12 18:26:44.002387 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.002342    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f56gm" event={"ID":"a28fc4fa-db9a-4279-aa00-fedb855f66a3","Type":"ContainerStarted","Data":"9044a30d4da950bea0d4aa9967f2dadaf3e2af2d0cac0f072bdf78332c53371d"}
Jul 12 18:26:44.002522 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.002404    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f56gm" event={"ID":"a28fc4fa-db9a-4279-aa00-fedb855f66a3","Type":"ContainerStarted","Data":"158408fdea051c7f8c2bd6059747cc4f0b82a6cb5b38a1004daeac11ef7b7240"}
Jul 12 18:26:44.021421 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.021381    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f56gm" podStartSLOduration=2.996764273 podStartE2EDuration="4.02136643s" totalImagesPullingTime="962.474201ms" totalInitContainerRuntime="62.127956ms" isStatefulPod=true podCreationTimestamp="2026-07-12 18:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:44.020253439 +0000 UTC m=+86.903603715" watchObservedRunningTime="2026-07-12 18:26:44.02136643 +0000 UTC m=+86.904716704"
Jul 12 18:26:44.235243 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.235221    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:26:44.303014 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.302928    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/96c3b950-1121-433a-9eff-8dfa9a0034a0-ca-trust-extracted\") pod \"96c3b950-1121-433a-9eff-8dfa9a0034a0\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") "
Jul 12 18:26:44.303014 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.302967    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-trusted-ca\") pod \"96c3b950-1121-433a-9eff-8dfa9a0034a0\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") "
Jul 12 18:26:44.303014 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.303012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-bound-sa-token\") pod \"96c3b950-1121-433a-9eff-8dfa9a0034a0\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") "
Jul 12 18:26:44.303618 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.303029    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-installation-pull-secrets\") pod \"96c3b950-1121-433a-9eff-8dfa9a0034a0\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") "
Jul 12 18:26:44.303618 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.303048    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls\") pod \"96c3b950-1121-433a-9eff-8dfa9a0034a0\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") "
Jul 12 18:26:44.303618 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.303079    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-image-registry-private-configuration\") pod \"96c3b950-1121-433a-9eff-8dfa9a0034a0\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") "
Jul 12 18:26:44.303618 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.303245    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjh2d\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-kube-api-access-jjh2d\") pod \"96c3b950-1121-433a-9eff-8dfa9a0034a0\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") "
Jul 12 18:26:44.303820 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.303617    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "96c3b950-1121-433a-9eff-8dfa9a0034a0" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:26:44.305715 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.305685    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "96c3b950-1121-433a-9eff-8dfa9a0034a0" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:26:44.305834 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.305799    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-kube-api-access-jjh2d" (OuterVolumeSpecName: "kube-api-access-jjh2d") pod "96c3b950-1121-433a-9eff-8dfa9a0034a0" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0"). InnerVolumeSpecName "kube-api-access-jjh2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:26:44.305834 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.305811    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "96c3b950-1121-433a-9eff-8dfa9a0034a0" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:26:44.305917 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.305850    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "96c3b950-1121-433a-9eff-8dfa9a0034a0" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:26:44.305917 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.305897    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "96c3b950-1121-433a-9eff-8dfa9a0034a0" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:26:44.312164 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.312141    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c3b950-1121-433a-9eff-8dfa9a0034a0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "96c3b950-1121-433a-9eff-8dfa9a0034a0" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:26:44.404114 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.404078    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-certificates\") pod \"96c3b950-1121-433a-9eff-8dfa9a0034a0\" (UID: \"96c3b950-1121-433a-9eff-8dfa9a0034a0\") "
Jul 12 18:26:44.404257 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.404193    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-bound-sa-token\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:26:44.404257 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.404204    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-installation-pull-secrets\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:26:44.404257 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.404214    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:26:44.404257 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.404223    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/96c3b950-1121-433a-9eff-8dfa9a0034a0-image-registry-private-configuration\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:26:44.404257 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.404231    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjh2d\" (UniqueName: \"kubernetes.io/projected/96c3b950-1121-433a-9eff-8dfa9a0034a0-kube-api-access-jjh2d\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:26:44.404257 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.404239    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/96c3b950-1121-433a-9eff-8dfa9a0034a0-ca-trust-extracted\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:26:44.404257 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.404247    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-trusted-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:26:44.404512 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.404425    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "96c3b950-1121-433a-9eff-8dfa9a0034a0" (UID: "96c3b950-1121-433a-9eff-8dfa9a0034a0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:26:44.504675 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:44.504641    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/96c3b950-1121-433a-9eff-8dfa9a0034a0-registry-certificates\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:26:45.006548 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.006511    2562 generic.go:356] "Generic (PLEG): container finished" podID="96c3b950-1121-433a-9eff-8dfa9a0034a0" containerID="ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52" exitCode=0
Jul 12 18:26:45.006715 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.006558    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44"
Jul 12 18:26:45.006715 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.006592    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44" event={"ID":"96c3b950-1121-433a-9eff-8dfa9a0034a0","Type":"ContainerDied","Data":"ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52"}
Jul 12 18:26:45.006715 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.006644    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcbcb4984-kjk44" event={"ID":"96c3b950-1121-433a-9eff-8dfa9a0034a0","Type":"ContainerDied","Data":"3c20e62445547b851488bd5d5e984de3713d75f0e061fcdd5327e286e3dbb36d"}
Jul 12 18:26:45.006715 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.006665    2562 scope.go:117] "RemoveContainer" containerID="ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52"
Jul 12 18:26:45.015988 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.015968    2562 scope.go:117] "RemoveContainer" containerID="ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52"
Jul 12 18:26:45.016608 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:26:45.016340    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52\": container with ID starting with ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52 not found: ID does not exist" containerID="ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52"
Jul 12 18:26:45.016608 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.016380    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52"} err="failed to get container status \"ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52\": rpc error: code = NotFound desc = could not find container \"ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52\": container with ID starting with ad9b61a2d65eb3f7d9ade2e6a145148fac61a9be2f2ef5d4c1416fd03b1c5f52 not found: ID does not exist"
Jul 12 18:26:45.029980 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.029955    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dcbcb4984-kjk44"]
Jul 12 18:26:45.035309 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.035282    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6dcbcb4984-kjk44"]
Jul 12 18:26:45.063880 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.063839    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:45.138055 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.138024    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:45.245198 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.245168    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:45.332135 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.332053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:45.601646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:45.601572    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96c3b950-1121-433a-9eff-8dfa9a0034a0" path="/var/lib/kubelet/pods/96c3b950-1121-433a-9eff-8dfa9a0034a0/volumes"
Jul 12 18:26:46.667529 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:46.667500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:48.986521 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:48.986484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:49.076414 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:49.076379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:52.296587 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:52.296551    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:52.354670 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:52.354635    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:54.109480 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:54.109430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:26:54.160513 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:26:54.160487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:00.950005 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:00.949978    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q77wh"
Jul 12 18:27:02.870823 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:02.870792    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:02.971648 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:02.971614    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:03.033212 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:03.033184    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:07.291627 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:07.291593    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:07.392208 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:07.392173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:12.823646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:12.823612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:12.875516 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:12.875449    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:18.146377 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:18.146348    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7msr2_2be4bca4-2772-4c13-a24b-eebc9d61cc7d/serve-healthcheck-canary/0.log"
Jul 12 18:27:29.511269 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:29.511163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:29.611328 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:29.611300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:29.662551 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:29.662521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:33.409627 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:33.409591    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:56.444707 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:56.444675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:27:56.542378 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:27:56.542349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:02.977627 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:02.977582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:03.027839 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:03.027808    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:08.368717 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:08.368686    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:08.420609 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:08.420580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:14.317944 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:14.317866    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:14.415224 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:14.415191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:14.466228 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:14.466200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:22.301246 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:22.301208    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:22.399673 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:22.399640    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:38.453662 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:38.453631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:38.546646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:38.546615    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:48.366736 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:48.366695    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:28:48.424053 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:28:48.424007    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:05.100790 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:05.100751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:05.916571 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:05.916543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:08.708349 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:08.708311    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:08.809042 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:08.809011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:09.797527 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:09.797491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:09.939579 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:09.939544    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:11.014558 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:11.014530    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:11.109876 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:11.109843    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:13.220039 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:13.220000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:13.455180 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:13.455145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:15.506731 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:15.506694    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:15.556869 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:15.556815    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:19.110352 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:19.110316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:19.211175 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:19.211145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:21.421699 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:21.421668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:24.235020 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:24.234983    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:24.333866 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:24.333832    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:26.420196 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:26.420168    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:35.679737 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:35.679708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:35.780868 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:35.780837    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:41.715958 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:41.715866    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:41.812390 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:41.812363    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:42.334924 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.334895    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:42.621264 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.621192    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:42.728129 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.728100    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:42.746258 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.746228    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"]
Jul 12 18:29:42.746526 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.746512    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c3b950-1121-433a-9eff-8dfa9a0034a0" containerName="registry"
Jul 12 18:29:42.746578 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.746528    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c3b950-1121-433a-9eff-8dfa9a0034a0" containerName="registry"
Jul 12 18:29:42.746578 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.746567    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c3b950-1121-433a-9eff-8dfa9a0034a0" containerName="registry"
Jul 12 18:29:42.748355 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.748339    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.750586 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.750560    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 12 18:29:42.750586 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.750578    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 12 18:29:42.750885 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.750865    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d6g59\""
Jul 12 18:29:42.750990 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.750924    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 12 18:29:42.751109 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.751085    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 12 18:29:42.765239 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.765217    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"]
Jul 12 18:29:42.855148 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.855113    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.855148 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.855151    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7fff\" (UniqueName: \"kubernetes.io/projected/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-kube-api-access-x7fff\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.855367 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.855178    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.855367 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.855294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-manifests\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.855367 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.855325    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-tmp\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.874586 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.874506    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:42.956046 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.956013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-manifests\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.956195 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.956050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-tmp\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.956195 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.956081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.956195 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.956103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7fff\" (UniqueName: \"kubernetes.io/projected/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-kube-api-access-x7fff\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.956195 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.956123    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.956577 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.956553    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-manifests\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.958479 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.958432    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-tmp\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.958602 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.958585    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.958683 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.958665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.966106 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.966076    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7fff\" (UniqueName: \"kubernetes.io/projected/610c69ab-8066-4ed3-9b87-86c9a36b7dcf-kube-api-access-x7fff\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-qpzh2\" (UID: \"610c69ab-8066-4ed3-9b87-86c9a36b7dcf\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:42.974551 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:42.974528    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:43.058442 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:43.058415    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:43.081003 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:43.080977    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:43.181232 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:43.181205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:43.195906 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:43.195882    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"]
Jul 12 18:29:43.199197 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:29:43.199170    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610c69ab_8066_4ed3_9b87_86c9a36b7dcf.slice/crio-ca0ba5a344b6c64d77a5fe09f272e78049a0e977ee162105d366da7de6958181 WatchSource:0}: Error finding container ca0ba5a344b6c64d77a5fe09f272e78049a0e977ee162105d366da7de6958181: Status 404 returned error can't find the container with id ca0ba5a344b6c64d77a5fe09f272e78049a0e977ee162105d366da7de6958181
Jul 12 18:29:43.275664 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:43.275630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:43.590068 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:43.590028    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2" event={"ID":"610c69ab-8066-4ed3-9b87-86c9a36b7dcf","Type":"ContainerStarted","Data":"ca0ba5a344b6c64d77a5fe09f272e78049a0e977ee162105d366da7de6958181"}
Jul 12 18:29:46.608511 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:46.608450    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2" event={"ID":"610c69ab-8066-4ed3-9b87-86c9a36b7dcf","Type":"ContainerStarted","Data":"e6eff23925f0df6b62d4643293b676538912784623e2491b5f6925c84bfc34d4"}
Jul 12 18:29:47.612297 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:47.612264    2562 generic.go:356] "Generic (PLEG): container finished" podID="610c69ab-8066-4ed3-9b87-86c9a36b7dcf" containerID="e6eff23925f0df6b62d4643293b676538912784623e2491b5f6925c84bfc34d4" exitCode=0
Jul 12 18:29:47.612755 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:47.612326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2" event={"ID":"610c69ab-8066-4ed3-9b87-86c9a36b7dcf","Type":"ContainerDied","Data":"e6eff23925f0df6b62d4643293b676538912784623e2491b5f6925c84bfc34d4"}
Jul 12 18:29:48.617337 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:48.617301    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2" event={"ID":"610c69ab-8066-4ed3-9b87-86c9a36b7dcf","Type":"ContainerStarted","Data":"59bf4686a0033c166b348ade175994fb2f7214da869d7caf2c79cac8e3aadf8e"}
Jul 12 18:29:48.617828 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:48.617435    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:29:48.638033 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:48.637980    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2" podStartSLOduration=2.423241271 podStartE2EDuration="6.63796551s" totalImagesPullingTime="3.207947074s" totalInitContainerRuntime="1.006777165s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:29:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:29:48.63652942 +0000 UTC m=+271.519879696" watchObservedRunningTime="2026-07-12 18:29:48.63796551 +0000 UTC m=+271.521315783"
Jul 12 18:29:49.185491 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:49.185433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:50.631221 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:50.631193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:50.732493 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:50.732442    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:29:59.622889 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:29:59.622854    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-qpzh2"
Jul 12 18:30:02.343430 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.343391    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:02.456352 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.456314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:02.843850 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.843814    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:02.897574 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.897541    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"]
Jul 12 18:30:02.908720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.908693    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:02.911720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.911691    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 12 18:30:02.911844 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.911791    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 12 18:30:02.912442 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.912425    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 12 18:30:02.913017 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.912975    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gd7c4\""
Jul 12 18:30:02.915903 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.915799    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 12 18:30:02.922143 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:02.922112    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"]
Jul 12 18:30:03.005633 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.005587    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a264575-50da-40c7-a0e5-4133810b085d-tls-certs\") pod \"kube-auth-proxy-5dd95989b6-klbnw\" (UID: \"4a264575-50da-40c7-a0e5-4133810b085d\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.005633 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.005645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djv2t\" (UniqueName: \"kubernetes.io/projected/4a264575-50da-40c7-a0e5-4133810b085d-kube-api-access-djv2t\") pod \"kube-auth-proxy-5dd95989b6-klbnw\" (UID: \"4a264575-50da-40c7-a0e5-4133810b085d\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.005913 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.005783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a264575-50da-40c7-a0e5-4133810b085d-tmp\") pod \"kube-auth-proxy-5dd95989b6-klbnw\" (UID: \"4a264575-50da-40c7-a0e5-4133810b085d\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.107122 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.107027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a264575-50da-40c7-a0e5-4133810b085d-tls-certs\") pod \"kube-auth-proxy-5dd95989b6-klbnw\" (UID: \"4a264575-50da-40c7-a0e5-4133810b085d\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.107122 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.107070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djv2t\" (UniqueName: \"kubernetes.io/projected/4a264575-50da-40c7-a0e5-4133810b085d-kube-api-access-djv2t\") pod \"kube-auth-proxy-5dd95989b6-klbnw\" (UID: \"4a264575-50da-40c7-a0e5-4133810b085d\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.107122 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.107114    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a264575-50da-40c7-a0e5-4133810b085d-tmp\") pod \"kube-auth-proxy-5dd95989b6-klbnw\" (UID: \"4a264575-50da-40c7-a0e5-4133810b085d\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.107401 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:30:03.107201    2562 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 12 18:30:03.107401 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:30:03.107289    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a264575-50da-40c7-a0e5-4133810b085d-tls-certs podName:4a264575-50da-40c7-a0e5-4133810b085d nodeName:}" failed. No retries permitted until 2026-07-12 18:30:03.607269546 +0000 UTC m=+286.490619815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4a264575-50da-40c7-a0e5-4133810b085d-tls-certs") pod "kube-auth-proxy-5dd95989b6-klbnw" (UID: "4a264575-50da-40c7-a0e5-4133810b085d") : secret "kube-auth-proxy-tls" not found
Jul 12 18:30:03.109742 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.109709    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a264575-50da-40c7-a0e5-4133810b085d-tmp\") pod \"kube-auth-proxy-5dd95989b6-klbnw\" (UID: \"4a264575-50da-40c7-a0e5-4133810b085d\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.118896 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.118870    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djv2t\" (UniqueName: \"kubernetes.io/projected/4a264575-50da-40c7-a0e5-4133810b085d-kube-api-access-djv2t\") pod \"kube-auth-proxy-5dd95989b6-klbnw\" (UID: \"4a264575-50da-40c7-a0e5-4133810b085d\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.132905 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.132870    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:03.194905 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.194876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:03.299175 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.299142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:03.396825 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.396740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:03.502145 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.502116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:03.611378 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.611343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a264575-50da-40c7-a0e5-4133810b085d-tls-certs\") pod \"kube-auth-proxy-5dd95989b6-klbnw\" (UID: \"4a264575-50da-40c7-a0e5-4133810b085d\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.613741 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.613722    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a264575-50da-40c7-a0e5-4133810b085d-tls-certs\") pod \"kube-auth-proxy-5dd95989b6-klbnw\" (UID: \"4a264575-50da-40c7-a0e5-4133810b085d\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.629098 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.629048    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:03.818815 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.818778    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"
Jul 12 18:30:03.961352 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:03.961319    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw"]
Jul 12 18:30:03.965160 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:30:03.965136    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a264575_50da_40c7_a0e5_4133810b085d.slice/crio-5497ca7158af3ed8173a16036265c8cddc00bb5b136d1d28c1a4edbb4269d550 WatchSource:0}: Error finding container 5497ca7158af3ed8173a16036265c8cddc00bb5b136d1d28c1a4edbb4269d550: Status 404 returned error can't find the container with id 5497ca7158af3ed8173a16036265c8cddc00bb5b136d1d28c1a4edbb4269d550
Jul 12 18:30:04.064996 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.064961    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"]
Jul 12 18:30:04.067900 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.067868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:04.068431 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.068401    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.070939 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.070775    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 12 18:30:04.070939 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.070775    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 12 18:30:04.071245 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.071223    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 12 18:30:04.072152 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.071450    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jfm46\""
Jul 12 18:30:04.072152 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.071707    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 12 18:30:04.085938 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.085915    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"]
Jul 12 18:30:04.151034 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.150989    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:04.215397 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.215342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.215397 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.215404    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.215698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.215426    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-cacerts\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.215698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.215520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3310753b-10ed-4b7f-ae44-57d717c84d60-local-certs\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.215698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.215565    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.215698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.215599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgc2x\" (UniqueName: \"kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-kube-api-access-dgc2x\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.215698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.215647    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-token\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.269729 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.269700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:04.316805 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.316757    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.316805 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.316808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-cacerts\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.317064 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.316828    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3310753b-10ed-4b7f-ae44-57d717c84d60-local-certs\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.317064 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.316851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.317064 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.316877    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgc2x\" (UniqueName: \"kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-kube-api-access-dgc2x\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.317064 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.316924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-token\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.317064 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.316981    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.317621 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.317581    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.319731 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.319705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.319817 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.319737    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3310753b-10ed-4b7f-ae44-57d717c84d60-local-certs\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.319817 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.319795    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-cacerts\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.319886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.319858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.330066 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.329995    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-token\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.330166 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.330136    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgc2x\" (UniqueName: \"kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-kube-api-access-dgc2x\") pod \"istiod-openshift-gateway-7574459656-7mp9m\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.381970 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.381930    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:04.513538 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.513514    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"]
Jul 12 18:30:04.516095 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:30:04.516064    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3310753b_10ed_4b7f_ae44_57d717c84d60.slice/crio-2ee3700a5df202313cc2d6f37d40e76e45ecdbb38dba0e0735c0a29dc3e19eac WatchSource:0}: Error finding container 2ee3700a5df202313cc2d6f37d40e76e45ecdbb38dba0e0735c0a29dc3e19eac: Status 404 returned error can't find the container with id 2ee3700a5df202313cc2d6f37d40e76e45ecdbb38dba0e0735c0a29dc3e19eac
Jul 12 18:30:04.672280 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.672178    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m" event={"ID":"3310753b-10ed-4b7f-ae44-57d717c84d60","Type":"ContainerStarted","Data":"2ee3700a5df202313cc2d6f37d40e76e45ecdbb38dba0e0735c0a29dc3e19eac"}
Jul 12 18:30:04.673570 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:04.673536    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw" event={"ID":"4a264575-50da-40c7-a0e5-4133810b085d","Type":"ContainerStarted","Data":"5497ca7158af3ed8173a16036265c8cddc00bb5b136d1d28c1a4edbb4269d550"}
Jul 12 18:30:05.140360 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:05.140325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:06.682300 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:06.682253    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw" event={"ID":"4a264575-50da-40c7-a0e5-4133810b085d","Type":"ContainerStarted","Data":"06612afc0d6d50c572b0ebb1103ed85de491fe7fafda5c53768b99cc4f7c40d3"}
Jul 12 18:30:06.698386 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:06.698205    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dd95989b6-klbnw" podStartSLOduration=2.226143216 podStartE2EDuration="4.698188268s" totalImagesPullingTime="2.472045052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:30:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:30:06.697957709 +0000 UTC m=+289.581307986" watchObservedRunningTime="2026-07-12 18:30:06.698188268 +0000 UTC m=+289.581538544"
Jul 12 18:30:09.356495 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:09.356436    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 12 18:30:09.356793 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:09.356539    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 12 18:30:09.698094 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:09.697994    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m" event={"ID":"3310753b-10ed-4b7f-ae44-57d717c84d60","Type":"ContainerStarted","Data":"dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec"}
Jul 12 18:30:09.698272 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:09.698113    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:09.718014 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:09.717957    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m" podStartSLOduration=0.879952633 podStartE2EDuration="5.717940775s" totalImagesPullingTime="4.837988142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:30:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:30:09.716317167 +0000 UTC m=+292.599667440" watchObservedRunningTime="2026-07-12 18:30:09.717940775 +0000 UTC m=+292.601291049"
Jul 12 18:30:10.189865 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:10.189826    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:10.294229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:10.294180    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:10.703280 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:10.703250    2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-7mp9m container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 12 18:30:10.703666 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:10.703299    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m" podUID="3310753b-10ed-4b7f-ae44-57d717c84d60" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 12 18:30:10.877631 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:10.877597    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:11.218847 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.218804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:11.693115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.693080    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"]
Jul 12 18:30:11.696704 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.696685    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.700212 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.700194    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 12 18:30:11.700738 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.700725    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 12 18:30:11.708572 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.708555    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 12 18:30:11.708895 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.708879    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 12 18:30:11.708950 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.708891    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5nrc5\""
Jul 12 18:30:11.709432 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.709419    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 12 18:30:11.720987 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.720967    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"]
Jul 12 18:30:11.777407 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.777375    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a885e50f-a14d-420c-b95f-f6a01d5b2f84-manager-config\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.777407 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.777408    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a885e50f-a14d-420c-b95f-f6a01d5b2f84-metrics-cert\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.777628 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.777427    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a885e50f-a14d-420c-b95f-f6a01d5b2f84-cert\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.777628 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.777510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw7rw\" (UniqueName: \"kubernetes.io/projected/a885e50f-a14d-420c-b95f-f6a01d5b2f84-kube-api-access-xw7rw\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.878832 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.878802    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw7rw\" (UniqueName: \"kubernetes.io/projected/a885e50f-a14d-420c-b95f-f6a01d5b2f84-kube-api-access-xw7rw\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.879008 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.878856    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a885e50f-a14d-420c-b95f-f6a01d5b2f84-manager-config\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.879008 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.878920    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a885e50f-a14d-420c-b95f-f6a01d5b2f84-metrics-cert\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.879008 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.878956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a885e50f-a14d-420c-b95f-f6a01d5b2f84-cert\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.879403 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.879386    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a885e50f-a14d-420c-b95f-f6a01d5b2f84-manager-config\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.881321 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.881300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a885e50f-a14d-420c-b95f-f6a01d5b2f84-metrics-cert\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.881397 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.881376    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a885e50f-a14d-420c-b95f-f6a01d5b2f84-cert\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:11.887473 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:11.887437    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw7rw\" (UniqueName: \"kubernetes.io/projected/a885e50f-a14d-420c-b95f-f6a01d5b2f84-kube-api-access-xw7rw\") pod \"lws-controller-manager-7f877c4f85-4qbqm\" (UID: \"a885e50f-a14d-420c-b95f-f6a01d5b2f84\") " pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:12.006685 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:12.006653    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:12.149079 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:12.149046    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"]
Jul 12 18:30:12.154837 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:30:12.154804    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda885e50f_a14d_420c_b95f_f6a01d5b2f84.slice/crio-f0dba4f5ea7dc08661e13f66780f7eaea4afa3ce5fce15b18870f6e3f42f942d WatchSource:0}: Error finding container f0dba4f5ea7dc08661e13f66780f7eaea4afa3ce5fce15b18870f6e3f42f942d: Status 404 returned error can't find the container with id f0dba4f5ea7dc08661e13f66780f7eaea4afa3ce5fce15b18870f6e3f42f942d
Jul 12 18:30:12.687502 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:12.687439    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:12.709780 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:12.709743    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm" event={"ID":"a885e50f-a14d-420c-b95f-f6a01d5b2f84","Type":"ContainerStarted","Data":"f0dba4f5ea7dc08661e13f66780f7eaea4afa3ce5fce15b18870f6e3f42f942d"}
Jul 12 18:30:13.595791 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:13.595758    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:13.702423 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:13.702392    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:13.702976 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:13.702956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:13.928723 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:13.928603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:14.097128 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:14.097091    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:14.871345 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:14.871318    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:14.977700 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:14.977659    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:15.102044 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:15.101996    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:16.873115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:16.873078    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"]
Jul 12 18:30:16.873537 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:16.873255    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m" podUID="3310753b-10ed-4b7f-ae44-57d717c84d60" containerName="discovery" containerID="cri-o://dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec" gracePeriod=30
Jul 12 18:30:17.121756 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.121734    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:17.216062 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.215975    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3310753b-10ed-4b7f-ae44-57d717c84d60-local-certs\") pod \"3310753b-10ed-4b7f-ae44-57d717c84d60\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") "
Jul 12 18:30:17.216062 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.216009    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-ca-configmap\") pod \"3310753b-10ed-4b7f-ae44-57d717c84d60\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") "
Jul 12 18:30:17.216062 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.216042    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-kubeconfig\") pod \"3310753b-10ed-4b7f-ae44-57d717c84d60\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") "
Jul 12 18:30:17.216340 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.216069    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-dns-cert\") pod \"3310753b-10ed-4b7f-ae44-57d717c84d60\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") "
Jul 12 18:30:17.216340 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.216197    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgc2x\" (UniqueName: \"kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-kube-api-access-dgc2x\") pod \"3310753b-10ed-4b7f-ae44-57d717c84d60\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") "
Jul 12 18:30:17.216340 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.216232    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-token\") pod \"3310753b-10ed-4b7f-ae44-57d717c84d60\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") "
Jul 12 18:30:17.216340 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.216295    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-cacerts\") pod \"3310753b-10ed-4b7f-ae44-57d717c84d60\" (UID: \"3310753b-10ed-4b7f-ae44-57d717c84d60\") "
Jul 12 18:30:17.216546 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.216450    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3310753b-10ed-4b7f-ae44-57d717c84d60" (UID: "3310753b-10ed-4b7f-ae44-57d717c84d60"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:30:17.216617 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.216563    2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-ca-configmap\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:30:17.218549 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.218526    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3310753b-10ed-4b7f-ae44-57d717c84d60-local-certs" (OuterVolumeSpecName: "local-certs") pod "3310753b-10ed-4b7f-ae44-57d717c84d60" (UID: "3310753b-10ed-4b7f-ae44-57d717c84d60"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:30:17.218549 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.218534    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-cacerts" (OuterVolumeSpecName: "cacerts") pod "3310753b-10ed-4b7f-ae44-57d717c84d60" (UID: "3310753b-10ed-4b7f-ae44-57d717c84d60"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:30:17.218756 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.218726    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3310753b-10ed-4b7f-ae44-57d717c84d60" (UID: "3310753b-10ed-4b7f-ae44-57d717c84d60"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:30:17.218855 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.218804    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-token" (OuterVolumeSpecName: "istio-token") pod "3310753b-10ed-4b7f-ae44-57d717c84d60" (UID: "3310753b-10ed-4b7f-ae44-57d717c84d60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:30:17.218855 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.218810    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3310753b-10ed-4b7f-ae44-57d717c84d60" (UID: "3310753b-10ed-4b7f-ae44-57d717c84d60"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:30:17.219040 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.219023    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-kube-api-access-dgc2x" (OuterVolumeSpecName: "kube-api-access-dgc2x") pod "3310753b-10ed-4b7f-ae44-57d717c84d60" (UID: "3310753b-10ed-4b7f-ae44-57d717c84d60"). InnerVolumeSpecName "kube-api-access-dgc2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:30:17.317124 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.317075    2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-csr-dns-cert\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:30:17.317124 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.317115    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgc2x\" (UniqueName: \"kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-kube-api-access-dgc2x\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:30:17.317124 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.317128    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-token\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:30:17.317124 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.317136    2562 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-cacerts\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:30:17.317124 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.317144    2562 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3310753b-10ed-4b7f-ae44-57d717c84d60-local-certs\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:30:17.317418 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.317151    2562 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3310753b-10ed-4b7f-ae44-57d717c84d60-istio-kubeconfig\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:30:17.491234 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.491074    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:30:17.495292 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.495264    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:30:17.499487 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.499448    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 12 18:30:17.728316 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.728285    2562 generic.go:356] "Generic (PLEG): container finished" podID="3310753b-10ed-4b7f-ae44-57d717c84d60" containerID="dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec" exitCode=0
Jul 12 18:30:17.728522 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.728341    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"
Jul 12 18:30:17.728522 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.728358    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m" event={"ID":"3310753b-10ed-4b7f-ae44-57d717c84d60","Type":"ContainerDied","Data":"dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec"}
Jul 12 18:30:17.728522 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.728396    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m" event={"ID":"3310753b-10ed-4b7f-ae44-57d717c84d60","Type":"ContainerDied","Data":"2ee3700a5df202313cc2d6f37d40e76e45ecdbb38dba0e0735c0a29dc3e19eac"}
Jul 12 18:30:17.728522 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.728415    2562 scope.go:117] "RemoveContainer" containerID="dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec"
Jul 12 18:30:17.735917 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.735898    2562 scope.go:117] "RemoveContainer" containerID="dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec"
Jul 12 18:30:17.736146 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:30:17.736126    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec\": container with ID starting with dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec not found: ID does not exist" containerID="dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec"
Jul 12 18:30:17.736192 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.736155    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec"} err="failed to get container status \"dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec\": rpc error: code = NotFound desc = could not find container \"dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec\": container with ID starting with dbb9e66fede81ec5846d90204f1e8276d8829c100498abe8fd4d6580e05f28ec not found: ID does not exist"
Jul 12 18:30:17.746780 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.746714    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"]
Jul 12 18:30:17.750838 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:17.750819    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7mp9m"]
Jul 12 18:30:19.601393 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:19.601351    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3310753b-10ed-4b7f-ae44-57d717c84d60" path="/var/lib/kubelet/pods/3310753b-10ed-4b7f-ae44-57d717c84d60/volumes"
Jul 12 18:30:43.822205 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:43.822171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm" event={"ID":"a885e50f-a14d-420c-b95f-f6a01d5b2f84","Type":"ContainerStarted","Data":"d50aceb6f3ee2ae5918873a3db9072bbea243132a974ff7e635b69c1d452d2e6"}
Jul 12 18:30:43.822622 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:43.822293    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:30:43.838663 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:43.838621    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm" podStartSLOduration=1.92157074 podStartE2EDuration="32.838608494s" totalImagesPullingTime="30.917037754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:30:43.837405168 +0000 UTC m=+326.720755442" watchObservedRunningTime="2026-07-12 18:30:43.838608494 +0000 UTC m=+326.721958770"
Jul 12 18:30:44.193753 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:44.193672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:44.250691 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:44.250659    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:30:54.829008 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:30:54.828976    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7f877c4f85-4qbqm"
Jul 12 18:31:03.806057 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:03.806027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:03.886004 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:03.885972    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:03.956577 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:03.956536    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:10.219630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:10.219521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:32.239838 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:32.239800    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:32.800984 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:32.800956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:37.208096 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:37.208066    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:39.458040 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:39.458006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:39.526776 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:39.526746    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:42.117261 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:42.117232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:42.447713 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:42.447615    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:44.817029 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:44.816984    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:45.466661 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:45.466630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:46.952264 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:46.952233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:47.049400 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:47.049367    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:47.865163 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:47.865126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:48.609705 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:48.609668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:48.666964 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:48.666923    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:49.017087 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:49.017060    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:49.081131 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:49.081093    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:50.699746 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.699713    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"]
Jul 12 18:31:50.700170 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.700080    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3310753b-10ed-4b7f-ae44-57d717c84d60" containerName="discovery"
Jul 12 18:31:50.700170 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.700093    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3310753b-10ed-4b7f-ae44-57d717c84d60" containerName="discovery"
Jul 12 18:31:50.700243 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.700173    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3310753b-10ed-4b7f-ae44-57d717c84d60" containerName="discovery"
Jul 12 18:31:50.703249 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.703231    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:50.705778 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.705756    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 12 18:31:50.705912 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.705777    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 12 18:31:50.705912 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.705822    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 12 18:31:50.706428 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.706411    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2dvvh\""
Jul 12 18:31:50.706573 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.706412    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 12 18:31:50.710130 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.710106    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"]
Jul 12 18:31:50.714271 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.714234    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:50.796933 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.796901    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d903caac-eeb0-479e-80b6-ad2b8b5b069b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z82qs\" (UID: \"d903caac-eeb0-479e-80b6-ad2b8b5b069b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:50.797133 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.796958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd868\" (UniqueName: \"kubernetes.io/projected/d903caac-eeb0-479e-80b6-ad2b8b5b069b-kube-api-access-dd868\") pod \"kuadrant-console-plugin-66b5774985-z82qs\" (UID: \"d903caac-eeb0-479e-80b6-ad2b8b5b069b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:50.797133 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.797034    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d903caac-eeb0-479e-80b6-ad2b8b5b069b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z82qs\" (UID: \"d903caac-eeb0-479e-80b6-ad2b8b5b069b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:50.897395 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.897359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd868\" (UniqueName: \"kubernetes.io/projected/d903caac-eeb0-479e-80b6-ad2b8b5b069b-kube-api-access-dd868\") pod \"kuadrant-console-plugin-66b5774985-z82qs\" (UID: \"d903caac-eeb0-479e-80b6-ad2b8b5b069b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:50.897609 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.897405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d903caac-eeb0-479e-80b6-ad2b8b5b069b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z82qs\" (UID: \"d903caac-eeb0-479e-80b6-ad2b8b5b069b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:50.897609 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.897435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d903caac-eeb0-479e-80b6-ad2b8b5b069b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z82qs\" (UID: \"d903caac-eeb0-479e-80b6-ad2b8b5b069b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:50.897609 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:31:50.897551    2562 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 12 18:31:50.897773 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:31:50.897630    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d903caac-eeb0-479e-80b6-ad2b8b5b069b-plugin-serving-cert podName:d903caac-eeb0-479e-80b6-ad2b8b5b069b nodeName:}" failed. No retries permitted until 2026-07-12 18:31:51.397608845 +0000 UTC m=+394.280959099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/d903caac-eeb0-479e-80b6-ad2b8b5b069b-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-z82qs" (UID: "d903caac-eeb0-479e-80b6-ad2b8b5b069b") : secret "plugin-serving-cert" not found
Jul 12 18:31:50.898113 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.898091    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d903caac-eeb0-479e-80b6-ad2b8b5b069b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z82qs\" (UID: \"d903caac-eeb0-479e-80b6-ad2b8b5b069b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:50.905627 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:50.905605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd868\" (UniqueName: \"kubernetes.io/projected/d903caac-eeb0-479e-80b6-ad2b8b5b069b-kube-api-access-dd868\") pod \"kuadrant-console-plugin-66b5774985-z82qs\" (UID: \"d903caac-eeb0-479e-80b6-ad2b8b5b069b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:51.111415 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:51.111377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:51.308286 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:51.308257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:51.400643 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:51.400564    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d903caac-eeb0-479e-80b6-ad2b8b5b069b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z82qs\" (UID: \"d903caac-eeb0-479e-80b6-ad2b8b5b069b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:51.402919 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:51.402897    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d903caac-eeb0-479e-80b6-ad2b8b5b069b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z82qs\" (UID: \"d903caac-eeb0-479e-80b6-ad2b8b5b069b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:51.403762 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:51.403735    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:51.613020 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:51.612988    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"
Jul 12 18:31:51.735993 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:51.735973    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs"]
Jul 12 18:31:51.738729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:31:51.738701    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd903caac_eeb0_479e_80b6_ad2b8b5b069b.slice/crio-a1477b112b2d24b8e7dcd8d77b658de1ae2b44969100a5f79ac2b7bece39a3e0 WatchSource:0}: Error finding container a1477b112b2d24b8e7dcd8d77b658de1ae2b44969100a5f79ac2b7bece39a3e0: Status 404 returned error can't find the container with id a1477b112b2d24b8e7dcd8d77b658de1ae2b44969100a5f79ac2b7bece39a3e0
Jul 12 18:31:51.740248 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:51.740233    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 18:31:52.062217 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:52.062180    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs" event={"ID":"d903caac-eeb0-479e-80b6-ad2b8b5b069b","Type":"ContainerStarted","Data":"a1477b112b2d24b8e7dcd8d77b658de1ae2b44969100a5f79ac2b7bece39a3e0"}
Jul 12 18:31:52.617000 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:52.616926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:52.716709 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:52.716655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:59.828561 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:59.828520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:31:59.924429 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:31:59.924398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:01.770009 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:01.769970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:01.870729 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:01.870690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:01.971001 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:01.970951    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:02.071364 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:02.071282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:03.844869 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:03.844829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:03.945686 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:03.945654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:07.624199 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:07.624154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:07.679117 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:07.679082    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:07.831030 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:07.830992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:07.885399 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:07.885299    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:10.301873 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:10.301840    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:10.401370 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:10.401326    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:13.150583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:13.150544    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs" event={"ID":"d903caac-eeb0-479e-80b6-ad2b8b5b069b","Type":"ContainerStarted","Data":"82cb98adeaba7788987cc77890a5ee1216819d08ae641da840fa7807f90bcf9b"}
Jul 12 18:32:13.165668 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:13.165609    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z82qs" podStartSLOduration=2.451751997 podStartE2EDuration="23.165594431s" totalImagesPullingTime="20.713842434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:31:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:32:13.164701601 +0000 UTC m=+416.048051875" watchObservedRunningTime="2026-07-12 18:32:13.165594431 +0000 UTC m=+416.048944704"
Jul 12 18:32:16.796014 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:16.795981    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:16.897194 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:16.897165    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:21.521866 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:21.521819    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:21.624027 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:21.623993    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:25.336058 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:25.336020    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:25.436934 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:25.436896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:25.717494 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:25.717387    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:25.772756 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:25.772725    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:29.010158 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:29.010128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:29.250681 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:29.250648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:29.729732 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:29.729700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:29.797988 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:29.797955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:30.696846 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:30.696816    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:30.795607 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:30.795576    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:30.851230 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:30.851197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:31.489987 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.489957    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-trsvr"]
Jul 12 18:32:31.520849 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.520812    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:31.536294 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.536274    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-trsvr"]
Jul 12 18:32:31.536370 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.536352    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-trsvr"
Jul 12 18:32:31.538882 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.538860    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-frqdv\""
Jul 12 18:32:31.620668 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.620640    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:31.635812 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.635787    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sp9s\" (UniqueName: \"kubernetes.io/projected/5a6b87ee-e949-4af8-a15b-6cc302b5746c-kube-api-access-9sp9s\") pod \"authorino-58cd955675-trsvr\" (UID: \"5a6b87ee-e949-4af8-a15b-6cc302b5746c\") " pod="kuadrant-system/authorino-58cd955675-trsvr"
Jul 12 18:32:31.688502 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.688452    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-qkt99"]
Jul 12 18:32:31.691605 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.691580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qkt99"
Jul 12 18:32:31.699267 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.699246    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qkt99"]
Jul 12 18:32:31.737218 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.737195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sp9s\" (UniqueName: \"kubernetes.io/projected/5a6b87ee-e949-4af8-a15b-6cc302b5746c-kube-api-access-9sp9s\") pod \"authorino-58cd955675-trsvr\" (UID: \"5a6b87ee-e949-4af8-a15b-6cc302b5746c\") " pod="kuadrant-system/authorino-58cd955675-trsvr"
Jul 12 18:32:31.744598 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.744551    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sp9s\" (UniqueName: \"kubernetes.io/projected/5a6b87ee-e949-4af8-a15b-6cc302b5746c-kube-api-access-9sp9s\") pod \"authorino-58cd955675-trsvr\" (UID: \"5a6b87ee-e949-4af8-a15b-6cc302b5746c\") " pod="kuadrant-system/authorino-58cd955675-trsvr"
Jul 12 18:32:31.838036 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.837999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n26q\" (UniqueName: \"kubernetes.io/projected/fce2a8ee-8d76-4765-9863-1d707ae492cf-kube-api-access-6n26q\") pod \"authorino-75d556bd77-qkt99\" (UID: \"fce2a8ee-8d76-4765-9863-1d707ae492cf\") " pod="kuadrant-system/authorino-75d556bd77-qkt99"
Jul 12 18:32:31.844922 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.844897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-trsvr"
Jul 12 18:32:31.938947 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.938922    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n26q\" (UniqueName: \"kubernetes.io/projected/fce2a8ee-8d76-4765-9863-1d707ae492cf-kube-api-access-6n26q\") pod \"authorino-75d556bd77-qkt99\" (UID: \"fce2a8ee-8d76-4765-9863-1d707ae492cf\") " pod="kuadrant-system/authorino-75d556bd77-qkt99"
Jul 12 18:32:31.947796 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.947764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n26q\" (UniqueName: \"kubernetes.io/projected/fce2a8ee-8d76-4765-9863-1d707ae492cf-kube-api-access-6n26q\") pod \"authorino-75d556bd77-qkt99\" (UID: \"fce2a8ee-8d76-4765-9863-1d707ae492cf\") " pod="kuadrant-system/authorino-75d556bd77-qkt99"
Jul 12 18:32:31.968083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:31.968060    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-trsvr"]
Jul 12 18:32:31.970349 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:32:31.970327    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a6b87ee_e949_4af8_a15b_6cc302b5746c.slice/crio-abf60468ece6c0dc13829372c8686f8a2f35a821feac64dcf9243b87f8121bae WatchSource:0}: Error finding container abf60468ece6c0dc13829372c8686f8a2f35a821feac64dcf9243b87f8121bae: Status 404 returned error can't find the container with id abf60468ece6c0dc13829372c8686f8a2f35a821feac64dcf9243b87f8121bae
Jul 12 18:32:32.000262 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:32.000202    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qkt99"
Jul 12 18:32:32.123987 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:32.123964    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qkt99"]
Jul 12 18:32:32.126304 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:32:32.126273    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfce2a8ee_8d76_4765_9863_1d707ae492cf.slice/crio-c5af2d3b8ddb1b4b832ea26f613d2177223ce85518727cdc666675084e803647 WatchSource:0}: Error finding container c5af2d3b8ddb1b4b832ea26f613d2177223ce85518727cdc666675084e803647: Status 404 returned error can't find the container with id c5af2d3b8ddb1b4b832ea26f613d2177223ce85518727cdc666675084e803647
Jul 12 18:32:32.215682 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:32.215655    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-trsvr" event={"ID":"5a6b87ee-e949-4af8-a15b-6cc302b5746c","Type":"ContainerStarted","Data":"abf60468ece6c0dc13829372c8686f8a2f35a821feac64dcf9243b87f8121bae"}
Jul 12 18:32:32.216717 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:32.216695    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qkt99" event={"ID":"fce2a8ee-8d76-4765-9863-1d707ae492cf","Type":"ContainerStarted","Data":"c5af2d3b8ddb1b4b832ea26f613d2177223ce85518727cdc666675084e803647"}
Jul 12 18:32:36.234864 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:36.234823    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-trsvr" event={"ID":"5a6b87ee-e949-4af8-a15b-6cc302b5746c","Type":"ContainerStarted","Data":"1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80"}
Jul 12 18:32:36.236094 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:36.236071    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qkt99" event={"ID":"fce2a8ee-8d76-4765-9863-1d707ae492cf","Type":"ContainerStarted","Data":"c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d"}
Jul 12 18:32:36.248724 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:36.248668    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-trsvr" podStartSLOduration=1.7965818009999999 podStartE2EDuration="5.248655624s" totalImagesPullingTime="3.452073823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:32:36.248434543 +0000 UTC m=+439.131784820" watchObservedRunningTime="2026-07-12 18:32:36.248655624 +0000 UTC m=+439.132005898"
Jul 12 18:32:36.261938 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:36.261875    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-qkt99" podStartSLOduration=1.955024761 podStartE2EDuration="5.26185609s" totalImagesPullingTime="3.306831329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:32:36.261521119 +0000 UTC m=+439.144871393" watchObservedRunningTime="2026-07-12 18:32:36.26185609 +0000 UTC m=+439.145206368"
Jul 12 18:32:36.286304 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:36.286272    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-trsvr"]
Jul 12 18:32:37.130853 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:37.130823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:37.180402 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:37.180364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:38.243321 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:38.243282    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-trsvr" podUID="5a6b87ee-e949-4af8-a15b-6cc302b5746c" containerName="authorino" containerID="cri-o://1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80" gracePeriod=30
Jul 12 18:32:38.476988 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:38.476963    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-trsvr"
Jul 12 18:32:38.598983 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:38.598888    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sp9s\" (UniqueName: \"kubernetes.io/projected/5a6b87ee-e949-4af8-a15b-6cc302b5746c-kube-api-access-9sp9s\") pod \"5a6b87ee-e949-4af8-a15b-6cc302b5746c\" (UID: \"5a6b87ee-e949-4af8-a15b-6cc302b5746c\") "
Jul 12 18:32:38.601104 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:38.601077    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a6b87ee-e949-4af8-a15b-6cc302b5746c-kube-api-access-9sp9s" (OuterVolumeSpecName: "kube-api-access-9sp9s") pod "5a6b87ee-e949-4af8-a15b-6cc302b5746c" (UID: "5a6b87ee-e949-4af8-a15b-6cc302b5746c"). InnerVolumeSpecName "kube-api-access-9sp9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:32:38.699302 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:38.699271    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sp9s\" (UniqueName: \"kubernetes.io/projected/5a6b87ee-e949-4af8-a15b-6cc302b5746c-kube-api-access-9sp9s\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:32:39.248179 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:39.248103    2562 generic.go:356] "Generic (PLEG): container finished" podID="5a6b87ee-e949-4af8-a15b-6cc302b5746c" containerID="1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80" exitCode=0
Jul 12 18:32:39.248179 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:39.248152    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-trsvr"
Jul 12 18:32:39.248179 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:39.248177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-trsvr" event={"ID":"5a6b87ee-e949-4af8-a15b-6cc302b5746c","Type":"ContainerDied","Data":"1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80"}
Jul 12 18:32:39.248677 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:39.248206    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-trsvr" event={"ID":"5a6b87ee-e949-4af8-a15b-6cc302b5746c","Type":"ContainerDied","Data":"abf60468ece6c0dc13829372c8686f8a2f35a821feac64dcf9243b87f8121bae"}
Jul 12 18:32:39.248677 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:39.248222    2562 scope.go:117] "RemoveContainer" containerID="1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80"
Jul 12 18:32:39.256506 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:39.256481    2562 scope.go:117] "RemoveContainer" containerID="1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80"
Jul 12 18:32:39.256793 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:32:39.256770    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80\": container with ID starting with 1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80 not found: ID does not exist" containerID="1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80"
Jul 12 18:32:39.256858 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:39.256801    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80"} err="failed to get container status \"1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80\": rpc error: code = NotFound desc = could not find container \"1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80\": container with ID starting with 1fd64068c9fe5518e37f552790f6f9be84677296ea9235b812bbefb02687dd80 not found: ID does not exist"
Jul 12 18:32:39.267559 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:39.267531    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-trsvr"]
Jul 12 18:32:39.273267 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:39.273246    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-trsvr"]
Jul 12 18:32:39.602688 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:39.602612    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a6b87ee-e949-4af8-a15b-6cc302b5746c" path="/var/lib/kubelet/pods/5a6b87ee-e949-4af8-a15b-6cc302b5746c/volumes"
Jul 12 18:32:40.933646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:40.933619    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:41.033861 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:41.033819    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:46.759360 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:46.759329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:46.858534 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:46.858499    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:32:47.295916 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:32:47.295886    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:00.296123 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.296087    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:00.516737 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.516702    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-csbpr"]
Jul 12 18:33:00.516968 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.516956    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a6b87ee-e949-4af8-a15b-6cc302b5746c" containerName="authorino"
Jul 12 18:33:00.516968 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.516969    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a6b87ee-e949-4af8-a15b-6cc302b5746c" containerName="authorino"
Jul 12 18:33:00.517072 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.517024    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a6b87ee-e949-4af8-a15b-6cc302b5746c" containerName="authorino"
Jul 12 18:33:00.573042 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.572957    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-csbpr"]
Jul 12 18:33:00.573210 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.573050    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-csbpr"
Jul 12 18:33:00.670763 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.670730    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdv7r\" (UniqueName: \"kubernetes.io/projected/836aae6f-a862-47d4-8eab-992cb8b1e86f-kube-api-access-wdv7r\") pod \"authorino-5d4b67c446-csbpr\" (UID: \"836aae6f-a862-47d4-8eab-992cb8b1e86f\") " pod="kuadrant-system/authorino-5d4b67c446-csbpr"
Jul 12 18:33:00.733942 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.733904    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-csbpr"]
Jul 12 18:33:00.734185 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:33:00.734162    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wdv7r], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-csbpr" podUID="836aae6f-a862-47d4-8eab-992cb8b1e86f"
Jul 12 18:33:00.766192 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.766158    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6bc46f89c8-t4j86"]
Jul 12 18:33:00.769395 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.769372    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bc46f89c8-t4j86"
Jul 12 18:33:00.771818 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.771796    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdv7r\" (UniqueName: \"kubernetes.io/projected/836aae6f-a862-47d4-8eab-992cb8b1e86f-kube-api-access-wdv7r\") pod \"authorino-5d4b67c446-csbpr\" (UID: \"836aae6f-a862-47d4-8eab-992cb8b1e86f\") " pod="kuadrant-system/authorino-5d4b67c446-csbpr"
Jul 12 18:33:00.775907 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.775882    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bc46f89c8-t4j86"]
Jul 12 18:33:00.781632 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.781611    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdv7r\" (UniqueName: \"kubernetes.io/projected/836aae6f-a862-47d4-8eab-992cb8b1e86f-kube-api-access-wdv7r\") pod \"authorino-5d4b67c446-csbpr\" (UID: \"836aae6f-a862-47d4-8eab-992cb8b1e86f\") " pod="kuadrant-system/authorino-5d4b67c446-csbpr"
Jul 12 18:33:00.872961 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.872868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jhgw\" (UniqueName: \"kubernetes.io/projected/e2fabf25-e9fe-488d-a193-ee4b1ce4964f-kube-api-access-5jhgw\") pod \"authorino-6bc46f89c8-t4j86\" (UID: \"e2fabf25-e9fe-488d-a193-ee4b1ce4964f\") " pod="kuadrant-system/authorino-6bc46f89c8-t4j86"
Jul 12 18:33:00.973366 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.973333    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jhgw\" (UniqueName: \"kubernetes.io/projected/e2fabf25-e9fe-488d-a193-ee4b1ce4964f-kube-api-access-5jhgw\") pod \"authorino-6bc46f89c8-t4j86\" (UID: \"e2fabf25-e9fe-488d-a193-ee4b1ce4964f\") " pod="kuadrant-system/authorino-6bc46f89c8-t4j86"
Jul 12 18:33:00.981593 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:00.981573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jhgw\" (UniqueName: \"kubernetes.io/projected/e2fabf25-e9fe-488d-a193-ee4b1ce4964f-kube-api-access-5jhgw\") pod \"authorino-6bc46f89c8-t4j86\" (UID: \"e2fabf25-e9fe-488d-a193-ee4b1ce4964f\") " pod="kuadrant-system/authorino-6bc46f89c8-t4j86"
Jul 12 18:33:01.078342 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:01.078308    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bc46f89c8-t4j86"
Jul 12 18:33:01.207677 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:01.207647    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bc46f89c8-t4j86"]
Jul 12 18:33:01.211251 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:33:01.211207    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2fabf25_e9fe_488d_a193_ee4b1ce4964f.slice/crio-dc6b7d39bb50323b60ad28b91941c2b65ff632114933bc64a28be5bf068b5d1c WatchSource:0}: Error finding container dc6b7d39bb50323b60ad28b91941c2b65ff632114933bc64a28be5bf068b5d1c: Status 404 returned error can't find the container with id dc6b7d39bb50323b60ad28b91941c2b65ff632114933bc64a28be5bf068b5d1c
Jul 12 18:33:01.312403 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:01.312371    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:01.339621 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:01.339593    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-csbpr"
Jul 12 18:33:01.339621 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:01.339609    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bc46f89c8-t4j86" event={"ID":"e2fabf25-e9fe-488d-a193-ee4b1ce4964f","Type":"ContainerStarted","Data":"dc6b7d39bb50323b60ad28b91941c2b65ff632114933bc64a28be5bf068b5d1c"}
Jul 12 18:33:01.347613 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:01.347591    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-csbpr"
Jul 12 18:33:01.376496 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:01.376406    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdv7r\" (UniqueName: \"kubernetes.io/projected/836aae6f-a862-47d4-8eab-992cb8b1e86f-kube-api-access-wdv7r\") pod \"836aae6f-a862-47d4-8eab-992cb8b1e86f\" (UID: \"836aae6f-a862-47d4-8eab-992cb8b1e86f\") "
Jul 12 18:33:01.378494 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:01.378450    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/836aae6f-a862-47d4-8eab-992cb8b1e86f-kube-api-access-wdv7r" (OuterVolumeSpecName: "kube-api-access-wdv7r") pod "836aae6f-a862-47d4-8eab-992cb8b1e86f" (UID: "836aae6f-a862-47d4-8eab-992cb8b1e86f"). InnerVolumeSpecName "kube-api-access-wdv7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:33:01.477298 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:01.477260    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdv7r\" (UniqueName: \"kubernetes.io/projected/836aae6f-a862-47d4-8eab-992cb8b1e86f-kube-api-access-wdv7r\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:33:02.345630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.345591    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bc46f89c8-t4j86" event={"ID":"e2fabf25-e9fe-488d-a193-ee4b1ce4964f","Type":"ContainerStarted","Data":"d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781"}
Jul 12 18:33:02.346115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.345605    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-csbpr"
Jul 12 18:33:02.360374 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.360316    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6bc46f89c8-t4j86" podStartSLOduration=1.9394719249999999 podStartE2EDuration="2.360301229s" totalImagesPullingTime="420.829304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:33:02.359576166 +0000 UTC m=+465.242926441" watchObservedRunningTime="2026-07-12 18:33:02.360301229 +0000 UTC m=+465.243651503"
Jul 12 18:33:02.386777 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.386742    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qkt99"]
Jul 12 18:33:02.386941 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.386920    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-qkt99" podUID="fce2a8ee-8d76-4765-9863-1d707ae492cf" containerName="authorino" containerID="cri-o://c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d" gracePeriod=30
Jul 12 18:33:02.389729 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.389697    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-csbpr"]
Jul 12 18:33:02.396452 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.396424    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-csbpr"]
Jul 12 18:33:02.606508 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.606413    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-576b45cd78-8tntf"]
Jul 12 18:33:02.614428 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.614397    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-576b45cd78-8tntf"
Jul 12 18:33:02.616774 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.616733    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 12 18:33:02.617881 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.617853    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-576b45cd78-8tntf"]
Jul 12 18:33:02.640746 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.640723    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qkt99"
Jul 12 18:33:02.687836 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.687798    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n26q\" (UniqueName: \"kubernetes.io/projected/fce2a8ee-8d76-4765-9863-1d707ae492cf-kube-api-access-6n26q\") pod \"fce2a8ee-8d76-4765-9863-1d707ae492cf\" (UID: \"fce2a8ee-8d76-4765-9863-1d707ae492cf\") "
Jul 12 18:33:02.687992 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.687978    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsmn8\" (UniqueName: \"kubernetes.io/projected/fd4879b9-fb77-486e-8b09-efa86eeb137a-kube-api-access-hsmn8\") pod \"authorino-576b45cd78-8tntf\" (UID: \"fd4879b9-fb77-486e-8b09-efa86eeb137a\") " pod="kuadrant-system/authorino-576b45cd78-8tntf"
Jul 12 18:33:02.688056 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.688006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd4879b9-fb77-486e-8b09-efa86eeb137a-tls-cert\") pod \"authorino-576b45cd78-8tntf\" (UID: \"fd4879b9-fb77-486e-8b09-efa86eeb137a\") " pod="kuadrant-system/authorino-576b45cd78-8tntf"
Jul 12 18:33:02.689795 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.689769    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fce2a8ee-8d76-4765-9863-1d707ae492cf-kube-api-access-6n26q" (OuterVolumeSpecName: "kube-api-access-6n26q") pod "fce2a8ee-8d76-4765-9863-1d707ae492cf" (UID: "fce2a8ee-8d76-4765-9863-1d707ae492cf"). InnerVolumeSpecName "kube-api-access-6n26q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:33:02.788951 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.788918    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsmn8\" (UniqueName: \"kubernetes.io/projected/fd4879b9-fb77-486e-8b09-efa86eeb137a-kube-api-access-hsmn8\") pod \"authorino-576b45cd78-8tntf\" (UID: \"fd4879b9-fb77-486e-8b09-efa86eeb137a\") " pod="kuadrant-system/authorino-576b45cd78-8tntf"
Jul 12 18:33:02.788951 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.788952    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd4879b9-fb77-486e-8b09-efa86eeb137a-tls-cert\") pod \"authorino-576b45cd78-8tntf\" (UID: \"fd4879b9-fb77-486e-8b09-efa86eeb137a\") " pod="kuadrant-system/authorino-576b45cd78-8tntf"
Jul 12 18:33:02.789137 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.788983    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n26q\" (UniqueName: \"kubernetes.io/projected/fce2a8ee-8d76-4765-9863-1d707ae492cf-kube-api-access-6n26q\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:33:02.791274 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.791254    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd4879b9-fb77-486e-8b09-efa86eeb137a-tls-cert\") pod \"authorino-576b45cd78-8tntf\" (UID: \"fd4879b9-fb77-486e-8b09-efa86eeb137a\") " pod="kuadrant-system/authorino-576b45cd78-8tntf"
Jul 12 18:33:02.796571 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.796552    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsmn8\" (UniqueName: \"kubernetes.io/projected/fd4879b9-fb77-486e-8b09-efa86eeb137a-kube-api-access-hsmn8\") pod \"authorino-576b45cd78-8tntf\" (UID: \"fd4879b9-fb77-486e-8b09-efa86eeb137a\") " pod="kuadrant-system/authorino-576b45cd78-8tntf"
Jul 12 18:33:02.937045 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:02.936957    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-576b45cd78-8tntf"
Jul 12 18:33:03.065784 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.065750    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-576b45cd78-8tntf"]
Jul 12 18:33:03.069505 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:33:03.069476    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd4879b9_fb77_486e_8b09_efa86eeb137a.slice/crio-16bfd9f76a8c99ecf80a6872f1544f34633221c93c51d87cbb5f8294333fcfd4 WatchSource:0}: Error finding container 16bfd9f76a8c99ecf80a6872f1544f34633221c93c51d87cbb5f8294333fcfd4: Status 404 returned error can't find the container with id 16bfd9f76a8c99ecf80a6872f1544f34633221c93c51d87cbb5f8294333fcfd4
Jul 12 18:33:03.350340 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.350305    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-576b45cd78-8tntf" event={"ID":"fd4879b9-fb77-486e-8b09-efa86eeb137a","Type":"ContainerStarted","Data":"16bfd9f76a8c99ecf80a6872f1544f34633221c93c51d87cbb5f8294333fcfd4"}
Jul 12 18:33:03.351479 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.351431    2562 generic.go:356] "Generic (PLEG): container finished" podID="fce2a8ee-8d76-4765-9863-1d707ae492cf" containerID="c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d" exitCode=0
Jul 12 18:33:03.351575 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.351488    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qkt99"
Jul 12 18:33:03.351575 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.351494    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qkt99" event={"ID":"fce2a8ee-8d76-4765-9863-1d707ae492cf","Type":"ContainerDied","Data":"c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d"}
Jul 12 18:33:03.351575 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.351535    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qkt99" event={"ID":"fce2a8ee-8d76-4765-9863-1d707ae492cf","Type":"ContainerDied","Data":"c5af2d3b8ddb1b4b832ea26f613d2177223ce85518727cdc666675084e803647"}
Jul 12 18:33:03.351575 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.351553    2562 scope.go:117] "RemoveContainer" containerID="c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d"
Jul 12 18:33:03.359762 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.359745    2562 scope.go:117] "RemoveContainer" containerID="c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d"
Jul 12 18:33:03.360021 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:33:03.360000    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d\": container with ID starting with c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d not found: ID does not exist" containerID="c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d"
Jul 12 18:33:03.360123 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.360026    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d"} err="failed to get container status \"c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d\": rpc error: code = NotFound desc = could not find container \"c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d\": container with ID starting with c0114e0ec805151525ecddaede19d98986d5f56489f7cc18d8bbdfaad854c16d not found: ID does not exist"
Jul 12 18:33:03.372504 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.372485    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qkt99"]
Jul 12 18:33:03.375007 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.374977    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qkt99"]
Jul 12 18:33:03.602296 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.602210    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="836aae6f-a862-47d4-8eab-992cb8b1e86f" path="/var/lib/kubelet/pods/836aae6f-a862-47d4-8eab-992cb8b1e86f/volumes"
Jul 12 18:33:03.602440 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:03.602429    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fce2a8ee-8d76-4765-9863-1d707ae492cf" path="/var/lib/kubelet/pods/fce2a8ee-8d76-4765-9863-1d707ae492cf/volumes"
Jul 12 18:33:04.356363 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:04.356326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-576b45cd78-8tntf" event={"ID":"fd4879b9-fb77-486e-8b09-efa86eeb137a","Type":"ContainerStarted","Data":"87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c"}
Jul 12 18:33:04.370297 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:04.370236    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-576b45cd78-8tntf" podStartSLOduration=2.091005611 podStartE2EDuration="2.370220849s" totalImagesPullingTime="279.215238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:33:04.369958059 +0000 UTC m=+467.253308333" watchObservedRunningTime="2026-07-12 18:33:04.370220849 +0000 UTC m=+467.253571122"
Jul 12 18:33:04.397377 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:04.397344    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bc46f89c8-t4j86"]
Jul 12 18:33:04.397557 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:04.397538    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6bc46f89c8-t4j86" podUID="e2fabf25-e9fe-488d-a193-ee4b1ce4964f" containerName="authorino" containerID="cri-o://d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781" gracePeriod=30
Jul 12 18:33:04.638332 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:04.638301    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bc46f89c8-t4j86"
Jul 12 18:33:04.703810 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:04.703776    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jhgw\" (UniqueName: \"kubernetes.io/projected/e2fabf25-e9fe-488d-a193-ee4b1ce4964f-kube-api-access-5jhgw\") pod \"e2fabf25-e9fe-488d-a193-ee4b1ce4964f\" (UID: \"e2fabf25-e9fe-488d-a193-ee4b1ce4964f\") "
Jul 12 18:33:04.705740 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:04.705716    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2fabf25-e9fe-488d-a193-ee4b1ce4964f-kube-api-access-5jhgw" (OuterVolumeSpecName: "kube-api-access-5jhgw") pod "e2fabf25-e9fe-488d-a193-ee4b1ce4964f" (UID: "e2fabf25-e9fe-488d-a193-ee4b1ce4964f"). InnerVolumeSpecName "kube-api-access-5jhgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:33:04.804251 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:04.804211    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jhgw\" (UniqueName: \"kubernetes.io/projected/e2fabf25-e9fe-488d-a193-ee4b1ce4964f-kube-api-access-5jhgw\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:33:05.362616 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:05.362582    2562 generic.go:356] "Generic (PLEG): container finished" podID="e2fabf25-e9fe-488d-a193-ee4b1ce4964f" containerID="d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781" exitCode=0
Jul 12 18:33:05.363049 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:05.362616    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bc46f89c8-t4j86" event={"ID":"e2fabf25-e9fe-488d-a193-ee4b1ce4964f","Type":"ContainerDied","Data":"d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781"}
Jul 12 18:33:05.363049 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:05.362642    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bc46f89c8-t4j86"
Jul 12 18:33:05.363049 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:05.362674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bc46f89c8-t4j86" event={"ID":"e2fabf25-e9fe-488d-a193-ee4b1ce4964f","Type":"ContainerDied","Data":"dc6b7d39bb50323b60ad28b91941c2b65ff632114933bc64a28be5bf068b5d1c"}
Jul 12 18:33:05.363049 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:05.362698    2562 scope.go:117] "RemoveContainer" containerID="d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781"
Jul 12 18:33:05.370744 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:05.370730    2562 scope.go:117] "RemoveContainer" containerID="d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781"
Jul 12 18:33:05.370988 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:33:05.370970    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781\": container with ID starting with d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781 not found: ID does not exist" containerID="d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781"
Jul 12 18:33:05.371035 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:05.370995    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781"} err="failed to get container status \"d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781\": rpc error: code = NotFound desc = could not find container \"d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781\": container with ID starting with d51712d6e7eff2b426717bb3dd30653ca91212d66edadac05c706522ae3e3781 not found: ID does not exist"
Jul 12 18:33:05.381727 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:05.381703    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bc46f89c8-t4j86"]
Jul 12 18:33:05.385388 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:05.385370    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6bc46f89c8-t4j86"]
Jul 12 18:33:05.601870 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:05.601840    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2fabf25-e9fe-488d-a193-ee4b1ce4964f" path="/var/lib/kubelet/pods/e2fabf25-e9fe-488d-a193-ee4b1ce4964f/volumes"
Jul 12 18:33:06.478471 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.478424    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:06.484342 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.479525    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8476d89dc8-45jlz"]
Jul 12 18:33:06.484342 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.480179    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fabf25-e9fe-488d-a193-ee4b1ce4964f" containerName="authorino"
Jul 12 18:33:06.484342 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.480196    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fabf25-e9fe-488d-a193-ee4b1ce4964f" containerName="authorino"
Jul 12 18:33:06.484342 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.480212    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fce2a8ee-8d76-4765-9863-1d707ae492cf" containerName="authorino"
Jul 12 18:33:06.484342 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.480221    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fce2a8ee-8d76-4765-9863-1d707ae492cf" containerName="authorino"
Jul 12 18:33:06.484342 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.480344    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fce2a8ee-8d76-4765-9863-1d707ae492cf" containerName="authorino"
Jul 12 18:33:06.484342 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.480361    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fabf25-e9fe-488d-a193-ee4b1ce4964f" containerName="authorino"
Jul 12 18:33:06.485758 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.485733    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:06.488228 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.488207    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wxgnx\""
Jul 12 18:33:06.488506 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.488482    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 12 18:33:06.488960 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.488943    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 12 18:33:06.493474 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.493357    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8476d89dc8-45jlz"]
Jul 12 18:33:06.516916 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.516891    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b850480-7d38-4303-b741-5253efc0024d-webhook-cert\") pod \"maas-controller-8476d89dc8-45jlz\" (UID: \"6b850480-7d38-4303-b741-5253efc0024d\") " pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:06.517053 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.516946    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkk9j\" (UniqueName: \"kubernetes.io/projected/6b850480-7d38-4303-b741-5253efc0024d-kube-api-access-wkk9j\") pod \"maas-controller-8476d89dc8-45jlz\" (UID: \"6b850480-7d38-4303-b741-5253efc0024d\") " pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:06.574396 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.574360    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:06.617752 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.617717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b850480-7d38-4303-b741-5253efc0024d-webhook-cert\") pod \"maas-controller-8476d89dc8-45jlz\" (UID: \"6b850480-7d38-4303-b741-5253efc0024d\") " pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:06.617892 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.617778    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkk9j\" (UniqueName: \"kubernetes.io/projected/6b850480-7d38-4303-b741-5253efc0024d-kube-api-access-wkk9j\") pod \"maas-controller-8476d89dc8-45jlz\" (UID: \"6b850480-7d38-4303-b741-5253efc0024d\") " pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:06.617892 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:33:06.617875    2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 12 18:33:06.617960 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:33:06.617954    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b850480-7d38-4303-b741-5253efc0024d-webhook-cert podName:6b850480-7d38-4303-b741-5253efc0024d nodeName:}" failed. No retries permitted until 2026-07-12 18:33:07.117934754 +0000 UTC m=+470.001285017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6b850480-7d38-4303-b741-5253efc0024d-webhook-cert") pod "maas-controller-8476d89dc8-45jlz" (UID: "6b850480-7d38-4303-b741-5253efc0024d") : secret "maas-controller-webhook-cert" not found
Jul 12 18:33:06.626954 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.626925    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkk9j\" (UniqueName: \"kubernetes.io/projected/6b850480-7d38-4303-b741-5253efc0024d-kube-api-access-wkk9j\") pod \"maas-controller-8476d89dc8-45jlz\" (UID: \"6b850480-7d38-4303-b741-5253efc0024d\") " pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:06.647255 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.647229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:06.953629 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:06.953550    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:07.122765 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:07.122735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b850480-7d38-4303-b741-5253efc0024d-webhook-cert\") pod \"maas-controller-8476d89dc8-45jlz\" (UID: \"6b850480-7d38-4303-b741-5253efc0024d\") " pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:07.125080 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:07.125059    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b850480-7d38-4303-b741-5253efc0024d-webhook-cert\") pod \"maas-controller-8476d89dc8-45jlz\" (UID: \"6b850480-7d38-4303-b741-5253efc0024d\") " pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:07.397385 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:07.397352    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:07.521183 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:07.521152    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8476d89dc8-45jlz"]
Jul 12 18:33:07.523999 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:33:07.523963    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b850480_7d38_4303_b741_5253efc0024d.slice/crio-0a4ce0fdc1a46a3a0a34686ec7d7b6bc602b2571a64898b09d79f2f2b851211d WatchSource:0}: Error finding container 0a4ce0fdc1a46a3a0a34686ec7d7b6bc602b2571a64898b09d79f2f2b851211d: Status 404 returned error can't find the container with id 0a4ce0fdc1a46a3a0a34686ec7d7b6bc602b2571a64898b09d79f2f2b851211d
Jul 12 18:33:08.377585 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:08.377549    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8476d89dc8-45jlz" event={"ID":"6b850480-7d38-4303-b741-5253efc0024d","Type":"ContainerStarted","Data":"0a4ce0fdc1a46a3a0a34686ec7d7b6bc602b2571a64898b09d79f2f2b851211d"}
Jul 12 18:33:09.770092 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:09.770006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:09.885221 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:09.885190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:10.387744 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:10.387711    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8476d89dc8-45jlz" event={"ID":"6b850480-7d38-4303-b741-5253efc0024d","Type":"ContainerStarted","Data":"9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9"}
Jul 12 18:33:10.387908 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:10.387824    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:10.414876 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:10.414828    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8476d89dc8-45jlz" podStartSLOduration=2.00610929 podStartE2EDuration="4.414807397s" totalImagesPullingTime="2.408698107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:33:10.413663776 +0000 UTC m=+473.297014053" watchObservedRunningTime="2026-07-12 18:33:10.414807397 +0000 UTC m=+473.298157671"
Jul 12 18:33:21.109394 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.109355    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8476d89dc8-45jlz"]
Jul 12 18:33:21.109792 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.109594    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8476d89dc8-45jlz" podUID="6b850480-7d38-4303-b741-5253efc0024d" containerName="manager" containerID="cri-o://9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9" gracePeriod=30
Jul 12 18:33:21.116770 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.116747    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:21.346278 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.346249    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:21.431337 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.431248    2562 generic.go:356] "Generic (PLEG): container finished" podID="6b850480-7d38-4303-b741-5253efc0024d" containerID="9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9" exitCode=0
Jul 12 18:33:21.431337 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.431301    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8476d89dc8-45jlz"
Jul 12 18:33:21.431337 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.431314    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8476d89dc8-45jlz" event={"ID":"6b850480-7d38-4303-b741-5253efc0024d","Type":"ContainerDied","Data":"9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9"}
Jul 12 18:33:21.431606 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.431361    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8476d89dc8-45jlz" event={"ID":"6b850480-7d38-4303-b741-5253efc0024d","Type":"ContainerDied","Data":"0a4ce0fdc1a46a3a0a34686ec7d7b6bc602b2571a64898b09d79f2f2b851211d"}
Jul 12 18:33:21.431606 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.431385    2562 scope.go:117] "RemoveContainer" containerID="9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9"
Jul 12 18:33:21.440360 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.440343    2562 scope.go:117] "RemoveContainer" containerID="9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9"
Jul 12 18:33:21.440650 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:33:21.440628    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9\": container with ID starting with 9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9 not found: ID does not exist" containerID="9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9"
Jul 12 18:33:21.440747 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.440658    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9"} err="failed to get container status \"9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9\": rpc error: code = NotFound desc = could not find container \"9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9\": container with ID starting with 9f6d9923ec68ecccba497886e865a17848d79b99441e0e30f7ca11b8058242f9 not found: ID does not exist"
Jul 12 18:33:21.440865 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.440852    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkk9j\" (UniqueName: \"kubernetes.io/projected/6b850480-7d38-4303-b741-5253efc0024d-kube-api-access-wkk9j\") pod \"6b850480-7d38-4303-b741-5253efc0024d\" (UID: \"6b850480-7d38-4303-b741-5253efc0024d\") "
Jul 12 18:33:21.440904 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.440880    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b850480-7d38-4303-b741-5253efc0024d-webhook-cert\") pod \"6b850480-7d38-4303-b741-5253efc0024d\" (UID: \"6b850480-7d38-4303-b741-5253efc0024d\") "
Jul 12 18:33:21.442958 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.442934    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b850480-7d38-4303-b741-5253efc0024d-kube-api-access-wkk9j" (OuterVolumeSpecName: "kube-api-access-wkk9j") pod "6b850480-7d38-4303-b741-5253efc0024d" (UID: "6b850480-7d38-4303-b741-5253efc0024d"). InnerVolumeSpecName "kube-api-access-wkk9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:33:21.443040 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.442968    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b850480-7d38-4303-b741-5253efc0024d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6b850480-7d38-4303-b741-5253efc0024d" (UID: "6b850480-7d38-4303-b741-5253efc0024d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:33:21.542352 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.542301    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkk9j\" (UniqueName: \"kubernetes.io/projected/6b850480-7d38-4303-b741-5253efc0024d-kube-api-access-wkk9j\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:33:21.542352 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.542339    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b850480-7d38-4303-b741-5253efc0024d-webhook-cert\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:33:21.745590 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.745557    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8476d89dc8-45jlz"]
Jul 12 18:33:21.751747 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:21.751723    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8476d89dc8-45jlz"]
Jul 12 18:33:23.605799 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:23.605768    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b850480-7d38-4303-b741-5253efc0024d" path="/var/lib/kubelet/pods/6b850480-7d38-4303-b741-5253efc0024d/volumes"
Jul 12 18:33:40.330964 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:40.330932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:40.431491 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:40.431443    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:44.153448 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:44.153416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:44.906081 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:44.906044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:45.011477 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.011433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:45.011690 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.011671    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"]
Jul 12 18:33:45.011989 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.011974    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b850480-7d38-4303-b741-5253efc0024d" containerName="manager"
Jul 12 18:33:45.012066 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.011991    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b850480-7d38-4303-b741-5253efc0024d" containerName="manager"
Jul 12 18:33:45.012118 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.012075    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b850480-7d38-4303-b741-5253efc0024d" containerName="manager"
Jul 12 18:33:45.015983 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.015963    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.018152 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.018129    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 12 18:33:45.018267 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.018209    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 12 18:33:45.018986 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.018967    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-j8nzl\""
Jul 12 18:33:45.019084 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.019045    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 12 18:33:45.030657 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.030632    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"]
Jul 12 18:33:45.131623 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.131589    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b77c272e-e419-4d28-89f5-45a5c249bb53-openshift-service-ca\") pod \"maas-api-b9ccf5cf-gfvl2\" (UID: \"b77c272e-e419-4d28-89f5-45a5c249bb53\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.131808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.131647    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b77c272e-e419-4d28-89f5-45a5c249bb53-maas-api-tls\") pod \"maas-api-b9ccf5cf-gfvl2\" (UID: \"b77c272e-e419-4d28-89f5-45a5c249bb53\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.131808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.131668    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7896\" (UniqueName: \"kubernetes.io/projected/b77c272e-e419-4d28-89f5-45a5c249bb53-kube-api-access-s7896\") pod \"maas-api-b9ccf5cf-gfvl2\" (UID: \"b77c272e-e419-4d28-89f5-45a5c249bb53\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.232705 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.232677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b77c272e-e419-4d28-89f5-45a5c249bb53-openshift-service-ca\") pod \"maas-api-b9ccf5cf-gfvl2\" (UID: \"b77c272e-e419-4d28-89f5-45a5c249bb53\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.233053 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.232737    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b77c272e-e419-4d28-89f5-45a5c249bb53-maas-api-tls\") pod \"maas-api-b9ccf5cf-gfvl2\" (UID: \"b77c272e-e419-4d28-89f5-45a5c249bb53\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.233053 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.232756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7896\" (UniqueName: \"kubernetes.io/projected/b77c272e-e419-4d28-89f5-45a5c249bb53-kube-api-access-s7896\") pod \"maas-api-b9ccf5cf-gfvl2\" (UID: \"b77c272e-e419-4d28-89f5-45a5c249bb53\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.233329 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.233307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b77c272e-e419-4d28-89f5-45a5c249bb53-openshift-service-ca\") pod \"maas-api-b9ccf5cf-gfvl2\" (UID: \"b77c272e-e419-4d28-89f5-45a5c249bb53\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.235130 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.235110    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b77c272e-e419-4d28-89f5-45a5c249bb53-maas-api-tls\") pod \"maas-api-b9ccf5cf-gfvl2\" (UID: \"b77c272e-e419-4d28-89f5-45a5c249bb53\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.247343 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.247321    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7896\" (UniqueName: \"kubernetes.io/projected/b77c272e-e419-4d28-89f5-45a5c249bb53-kube-api-access-s7896\") pod \"maas-api-b9ccf5cf-gfvl2\" (UID: \"b77c272e-e419-4d28-89f5-45a5c249bb53\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.327607 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.327564    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:45.367809 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.367775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:45.459956 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.459933    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"]
Jul 12 18:33:45.462529 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:33:45.462503    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb77c272e_e419_4d28_89f5_45a5c249bb53.slice/crio-074cf93b41581fb253bf1ea926c41591fcc02a7ae6789cf8d96add02ec39a9fc WatchSource:0}: Error finding container 074cf93b41581fb253bf1ea926c41591fcc02a7ae6789cf8d96add02ec39a9fc: Status 404 returned error can't find the container with id 074cf93b41581fb253bf1ea926c41591fcc02a7ae6789cf8d96add02ec39a9fc
Jul 12 18:33:45.463180 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.463152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:45.529517 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:45.529437    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2" event={"ID":"b77c272e-e419-4d28-89f5-45a5c249bb53","Type":"ContainerStarted","Data":"074cf93b41581fb253bf1ea926c41591fcc02a7ae6789cf8d96add02ec39a9fc"}
Jul 12 18:33:47.540034 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:47.540001    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2" event={"ID":"b77c272e-e419-4d28-89f5-45a5c249bb53","Type":"ContainerStarted","Data":"e61b44f20e54bcd5c059fbb9221d599c316fe8b99f6dc94d1b5a60b6a620da63"}
Jul 12 18:33:47.540420 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:47.540131    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:47.555886 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:47.555836    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2" podStartSLOduration=2.186761962 podStartE2EDuration="3.555818538s" totalImagesPullingTime="1.369056576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:33:47.554752636 +0000 UTC m=+510.438102912" watchObservedRunningTime="2026-07-12 18:33:47.555818538 +0000 UTC m=+510.439168811"
Jul 12 18:33:53.548627 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:53.548595    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-gfvl2"
Jul 12 18:33:54.551852 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:54.551820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:33:54.646878 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:33:54.646846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:34:02.348282 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:02.348246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:34:11.734950 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:11.734871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:34:22.802990 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:22.802956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:34:39.310010 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.309973    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"]
Jul 12 18:34:39.315266 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.315246    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.318166 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.318142    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 12 18:34:39.318296 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.318142    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 12 18:34:39.318296 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.318148    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6qzs6\""
Jul 12 18:34:39.318296 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.318176    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 12 18:34:39.322366 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.322173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"]
Jul 12 18:34:39.329946 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.329922    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:34:39.459587 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.459557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.459587 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.459590    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.459782 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.459655    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.459782 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.459673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.459782 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.459695    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gknw4\" (UniqueName: \"kubernetes.io/projected/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-kube-api-access-gknw4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.459782 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.459752    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.560583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.560428    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.560583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.560481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.560583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.560532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.560583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.560552    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.560583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.560583    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gknw4\" (UniqueName: \"kubernetes.io/projected/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-kube-api-access-gknw4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.560955 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.560624    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.561009 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.560970    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.561061 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.561009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.561061 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.561047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.562861 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.562837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.563120 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.563098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.568119 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.568094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gknw4\" (UniqueName: \"kubernetes.io/projected/a5f99b35-910c-4ff6-9f06-7a8a5747a49b-kube-api-access-gknw4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b\" (UID: \"a5f99b35-910c-4ff6-9f06-7a8a5747a49b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.626126 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.626097    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:39.751990 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:39.751843    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"]
Jul 12 18:34:39.754530 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:34:39.754503    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f99b35_910c_4ff6_9f06_7a8a5747a49b.slice/crio-efba967edf44f604de39e687676b4c618f9963cb9d0159063951c37c179117ac WatchSource:0}: Error finding container efba967edf44f604de39e687676b4c618f9963cb9d0159063951c37c179117ac: Status 404 returned error can't find the container with id efba967edf44f604de39e687676b4c618f9963cb9d0159063951c37c179117ac
Jul 12 18:34:40.743439 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:40.743415    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b" event={"ID":"a5f99b35-910c-4ff6-9f06-7a8a5747a49b","Type":"ContainerStarted","Data":"efba967edf44f604de39e687676b4c618f9963cb9d0159063951c37c179117ac"}
Jul 12 18:34:41.749409 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:41.749375    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b" event={"ID":"a5f99b35-910c-4ff6-9f06-7a8a5747a49b","Type":"ContainerStarted","Data":"0b7dcf19f60aa7a39e7e6fd16b54ecba0a6031e8e96a942aceccc236f76cd09f"}
Jul 12 18:34:41.767575 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:41.767526    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b" podStartSLOduration=1.802871176 podStartE2EDuration="2.767510058s" totalImagesPullingTime="964.638882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:34:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:34:41.766229066 +0000 UTC m=+564.649579336" watchObservedRunningTime="2026-07-12 18:34:41.767510058 +0000 UTC m=+564.650860331"
Jul 12 18:34:49.626420 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:49.626387    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:49.626906 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:49.626449    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:49.638889 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:49.638863    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:34:49.791409 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:34:49.791381    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b"
Jul 12 18:35:03.014410 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.014371    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"]
Jul 12 18:35:03.021248 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.021227    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.023538 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.023518    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 12 18:35:03.027462 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.027435    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"]
Jul 12 18:35:03.044955 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.044927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:35:03.147532 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.147491    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.147698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.147551    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.147698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.147577    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhfxk\" (UniqueName: \"kubernetes.io/projected/1219589d-93c9-4ed4-bea9-e4b717156ffc-kube-api-access-dhfxk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.147698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.147598    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1219589d-93c9-4ed4-bea9-e4b717156ffc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.147698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.147620    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.147859 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.147723    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.248337 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.248301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.248337 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.248345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.248616 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.248363    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhfxk\" (UniqueName: \"kubernetes.io/projected/1219589d-93c9-4ed4-bea9-e4b717156ffc-kube-api-access-dhfxk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.248616 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.248391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1219589d-93c9-4ed4-bea9-e4b717156ffc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.248616 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.248419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.248616 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.248452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.248847 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.248821    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.248905 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.248868    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.248957 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.248922    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.250779 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.250755    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1219589d-93c9-4ed4-bea9-e4b717156ffc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.250943 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.250924    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1219589d-93c9-4ed4-bea9-e4b717156ffc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.258557 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.258535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhfxk\" (UniqueName: \"kubernetes.io/projected/1219589d-93c9-4ed4-bea9-e4b717156ffc-kube-api-access-dhfxk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q4cbr\" (UID: \"1219589d-93c9-4ed4-bea9-e4b717156ffc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.332227 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.332139    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:03.461971 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.461936    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"]
Jul 12 18:35:03.465840 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:35:03.465809    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1219589d_93c9_4ed4_bea9_e4b717156ffc.slice/crio-7b46287fd38e885c636f840e2549e5eb836ae9ecc768fd2e4400d0a3d3c3f4d4 WatchSource:0}: Error finding container 7b46287fd38e885c636f840e2549e5eb836ae9ecc768fd2e4400d0a3d3c3f4d4: Status 404 returned error can't find the container with id 7b46287fd38e885c636f840e2549e5eb836ae9ecc768fd2e4400d0a3d3c3f4d4
Jul 12 18:35:03.838182 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.838153    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr" event={"ID":"1219589d-93c9-4ed4-bea9-e4b717156ffc","Type":"ContainerStarted","Data":"916ab4e1a0d029bb053e185c15930b18063346244854d69fb9970803cad2b8e0"}
Jul 12 18:35:03.838362 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.838203    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr" event={"ID":"1219589d-93c9-4ed4-bea9-e4b717156ffc","Type":"ContainerStarted","Data":"7b46287fd38e885c636f840e2549e5eb836ae9ecc768fd2e4400d0a3d3c3f4d4"}
Jul 12 18:35:03.855759 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:03.855706    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr" podStartSLOduration=1.681518011 podStartE2EDuration="1.855691156s" totalImagesPullingTime="174.173145ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:35:03.854946104 +0000 UTC m=+586.738296390" watchObservedRunningTime="2026-07-12 18:35:03.855691156 +0000 UTC m=+586.739041430"
Jul 12 18:35:11.939950 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:11.939920    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:35:13.332587 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:13.332541    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:13.332998 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:13.332654    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:13.345040 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:13.345011    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:13.887747 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:13.887717    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q4cbr"
Jul 12 18:35:17.520692 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:17.520665    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:35:17.521626 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:17.521605    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:35:20.112126 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.112095    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"]
Jul 12 18:35:20.115202 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.115185    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.117326 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.117304    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 12 18:35:20.126839 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.126808    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"]
Jul 12 18:35:20.138810 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.138776    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:35:20.195173 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.195136    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29b9beae-472d-416e-aba7-aec223326920-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.195372 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.195189    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.195372 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.195218    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.195372 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.195286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.195372 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.195323    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.195372 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.195363    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng9dm\" (UniqueName: \"kubernetes.io/projected/29b9beae-472d-416e-aba7-aec223326920-kube-api-access-ng9dm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.296692 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.296653    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.296692 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.296695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.296924 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.296724    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.296924 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.296741    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.296924 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.296784    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng9dm\" (UniqueName: \"kubernetes.io/projected/29b9beae-472d-416e-aba7-aec223326920-kube-api-access-ng9dm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.296924 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.296829    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29b9beae-472d-416e-aba7-aec223326920-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.297144 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.297115    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.297273 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.297147    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.297273 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.297212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.299015 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.298996    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29b9beae-472d-416e-aba7-aec223326920-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.299132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.299114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29b9beae-472d-416e-aba7-aec223326920-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.304235 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.304212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng9dm\" (UniqueName: \"kubernetes.io/projected/29b9beae-472d-416e-aba7-aec223326920-kube-api-access-ng9dm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9\" (UID: \"29b9beae-472d-416e-aba7-aec223326920\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.423678 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.423591    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:20.549584 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.549554    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"]
Jul 12 18:35:20.552729 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:35:20.552701    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29b9beae_472d_416e_aba7_aec223326920.slice/crio-d7a07cd5adb48c7fa553249984f5df9d0dcb563db4e83cd50d13643c39f14379 WatchSource:0}: Error finding container d7a07cd5adb48c7fa553249984f5df9d0dcb563db4e83cd50d13643c39f14379: Status 404 returned error can't find the container with id d7a07cd5adb48c7fa553249984f5df9d0dcb563db4e83cd50d13643c39f14379
Jul 12 18:35:20.910976 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.910938    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9" event={"ID":"29b9beae-472d-416e-aba7-aec223326920","Type":"ContainerStarted","Data":"d9ae1f761990cd0c2f3b901831ad8169164021291bdfab12beb09d9624d34800"}
Jul 12 18:35:20.910976 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.910981    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9" event={"ID":"29b9beae-472d-416e-aba7-aec223326920","Type":"ContainerStarted","Data":"d7a07cd5adb48c7fa553249984f5df9d0dcb563db4e83cd50d13643c39f14379"}
Jul 12 18:35:20.928615 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:20.928566    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9" podStartSLOduration=0.74678013 podStartE2EDuration="928.551891ms" totalImagesPullingTime="181.771761ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:35:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:35:20.927028606 +0000 UTC m=+603.810378883" watchObservedRunningTime="2026-07-12 18:35:20.928551891 +0000 UTC m=+603.811902165"
Jul 12 18:35:30.424180 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:30.424146    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:30.424589 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:30.424198    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:30.436757 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:30.436734    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:30.959250 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:30.959216    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9"
Jul 12 18:35:47.325503 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:47.325402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:35:47.426505 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:47.426476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:35:55.030304 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:35:55.030274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:36:16.804531 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:16.804495    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:36:16.854919 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:16.854891    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:36:21.513863 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.513824    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b6bb9c74c-lblfj"]
Jul 12 18:36:21.516985 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.516968    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6bb9c74c-lblfj"
Jul 12 18:36:21.525438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.525412    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b6bb9c74c-lblfj"]
Jul 12 18:36:21.615276 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.615240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgb7m\" (UniqueName: \"kubernetes.io/projected/49bd55f6-05cf-473a-baa4-bd4ae76a9ecb-kube-api-access-jgb7m\") pod \"authorino-7b6bb9c74c-lblfj\" (UID: \"49bd55f6-05cf-473a-baa4-bd4ae76a9ecb\") " pod="kuadrant-system/authorino-7b6bb9c74c-lblfj"
Jul 12 18:36:21.615437 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.615305    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49bd55f6-05cf-473a-baa4-bd4ae76a9ecb-tls-cert\") pod \"authorino-7b6bb9c74c-lblfj\" (UID: \"49bd55f6-05cf-473a-baa4-bd4ae76a9ecb\") " pod="kuadrant-system/authorino-7b6bb9c74c-lblfj"
Jul 12 18:36:21.716149 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.716118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgb7m\" (UniqueName: \"kubernetes.io/projected/49bd55f6-05cf-473a-baa4-bd4ae76a9ecb-kube-api-access-jgb7m\") pod \"authorino-7b6bb9c74c-lblfj\" (UID: \"49bd55f6-05cf-473a-baa4-bd4ae76a9ecb\") " pod="kuadrant-system/authorino-7b6bb9c74c-lblfj"
Jul 12 18:36:21.716299 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.716174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49bd55f6-05cf-473a-baa4-bd4ae76a9ecb-tls-cert\") pod \"authorino-7b6bb9c74c-lblfj\" (UID: \"49bd55f6-05cf-473a-baa4-bd4ae76a9ecb\") " pod="kuadrant-system/authorino-7b6bb9c74c-lblfj"
Jul 12 18:36:21.718518 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.718495    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49bd55f6-05cf-473a-baa4-bd4ae76a9ecb-tls-cert\") pod \"authorino-7b6bb9c74c-lblfj\" (UID: \"49bd55f6-05cf-473a-baa4-bd4ae76a9ecb\") " pod="kuadrant-system/authorino-7b6bb9c74c-lblfj"
Jul 12 18:36:21.724174 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.724146    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgb7m\" (UniqueName: \"kubernetes.io/projected/49bd55f6-05cf-473a-baa4-bd4ae76a9ecb-kube-api-access-jgb7m\") pod \"authorino-7b6bb9c74c-lblfj\" (UID: \"49bd55f6-05cf-473a-baa4-bd4ae76a9ecb\") " pod="kuadrant-system/authorino-7b6bb9c74c-lblfj"
Jul 12 18:36:21.825732 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.825656    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6bb9c74c-lblfj"
Jul 12 18:36:21.951395 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:21.951367    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b6bb9c74c-lblfj"]
Jul 12 18:36:21.953883 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:36:21.953856    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49bd55f6_05cf_473a_baa4_bd4ae76a9ecb.slice/crio-fd736ea377cd297b986d88850f3351aa75e9fdf86b615f5508698822f64b46b3 WatchSource:0}: Error finding container fd736ea377cd297b986d88850f3351aa75e9fdf86b615f5508698822f64b46b3: Status 404 returned error can't find the container with id fd736ea377cd297b986d88850f3351aa75e9fdf86b615f5508698822f64b46b3
Jul 12 18:36:22.149383 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:22.149291    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b6bb9c74c-lblfj" event={"ID":"49bd55f6-05cf-473a-baa4-bd4ae76a9ecb","Type":"ContainerStarted","Data":"fd736ea377cd297b986d88850f3351aa75e9fdf86b615f5508698822f64b46b3"}
Jul 12 18:36:23.156019 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.155980    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b6bb9c74c-lblfj" event={"ID":"49bd55f6-05cf-473a-baa4-bd4ae76a9ecb","Type":"ContainerStarted","Data":"019c048eb48abe1dde46e31b6b1408d472f0d44b9b4e0cac01ebf62a647a6fe9"}
Jul 12 18:36:23.170913 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.170852    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b6bb9c74c-lblfj" podStartSLOduration=1.693495744 podStartE2EDuration="2.170829688s" totalImagesPullingTime="477.333944ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:36:23.170377608 +0000 UTC m=+666.053727885" watchObservedRunningTime="2026-07-12 18:36:23.170829688 +0000 UTC m=+666.054179965"
Jul 12 18:36:23.200252 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.200215    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-576b45cd78-8tntf"]
Jul 12 18:36:23.200509 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.200481    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-576b45cd78-8tntf" podUID="fd4879b9-fb77-486e-8b09-efa86eeb137a" containerName="authorino" containerID="cri-o://87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c" gracePeriod=30
Jul 12 18:36:23.462850 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.462821    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-576b45cd78-8tntf"
Jul 12 18:36:23.631083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.631048    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd4879b9-fb77-486e-8b09-efa86eeb137a-tls-cert\") pod \"fd4879b9-fb77-486e-8b09-efa86eeb137a\" (UID: \"fd4879b9-fb77-486e-8b09-efa86eeb137a\") "
Jul 12 18:36:23.631241 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.631123    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsmn8\" (UniqueName: \"kubernetes.io/projected/fd4879b9-fb77-486e-8b09-efa86eeb137a-kube-api-access-hsmn8\") pod \"fd4879b9-fb77-486e-8b09-efa86eeb137a\" (UID: \"fd4879b9-fb77-486e-8b09-efa86eeb137a\") "
Jul 12 18:36:23.633526 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.633493    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4879b9-fb77-486e-8b09-efa86eeb137a-kube-api-access-hsmn8" (OuterVolumeSpecName: "kube-api-access-hsmn8") pod "fd4879b9-fb77-486e-8b09-efa86eeb137a" (UID: "fd4879b9-fb77-486e-8b09-efa86eeb137a"). InnerVolumeSpecName "kube-api-access-hsmn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:36:23.642356 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.642327    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd4879b9-fb77-486e-8b09-efa86eeb137a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "fd4879b9-fb77-486e-8b09-efa86eeb137a" (UID: "fd4879b9-fb77-486e-8b09-efa86eeb137a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:36:23.691115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.691067    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:36:23.732847 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.732573    2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd4879b9-fb77-486e-8b09-efa86eeb137a-tls-cert\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:36:23.732847 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.732604    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsmn8\" (UniqueName: \"kubernetes.io/projected/fd4879b9-fb77-486e-8b09-efa86eeb137a-kube-api-access-hsmn8\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:36:23.791083 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:23.791050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:36:24.160716 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.160626    2562 generic.go:356] "Generic (PLEG): container finished" podID="fd4879b9-fb77-486e-8b09-efa86eeb137a" containerID="87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c" exitCode=0
Jul 12 18:36:24.160716 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.160669    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-576b45cd78-8tntf"
Jul 12 18:36:24.161152 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.160713    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-576b45cd78-8tntf" event={"ID":"fd4879b9-fb77-486e-8b09-efa86eeb137a","Type":"ContainerDied","Data":"87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c"}
Jul 12 18:36:24.161152 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.160763    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-576b45cd78-8tntf" event={"ID":"fd4879b9-fb77-486e-8b09-efa86eeb137a","Type":"ContainerDied","Data":"16bfd9f76a8c99ecf80a6872f1544f34633221c93c51d87cbb5f8294333fcfd4"}
Jul 12 18:36:24.161152 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.160792    2562 scope.go:117] "RemoveContainer" containerID="87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c"
Jul 12 18:36:24.169821 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.169799    2562 scope.go:117] "RemoveContainer" containerID="87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c"
Jul 12 18:36:24.170107 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:36:24.170075    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c\": container with ID starting with 87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c not found: ID does not exist" containerID="87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c"
Jul 12 18:36:24.170185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.170120    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c"} err="failed to get container status \"87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c\": rpc error: code = NotFound desc = could not find container \"87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c\": container with ID starting with 87a182990bbd20d7acbaa78d1c9c11e69bcbb4664c25a7ca6880cad797dd145c not found: ID does not exist"
Jul 12 18:36:24.191113 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.191085    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-576b45cd78-8tntf"]
Jul 12 18:36:24.198533 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.198512    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-576b45cd78-8tntf"]
Jul 12 18:36:24.811871 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.811834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:36:24.910849 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:24.910818    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:36:25.602149 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:36:25.602121    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd4879b9-fb77-486e-8b09-efa86eeb137a" path="/var/lib/kubelet/pods/fd4879b9-fb77-486e-8b09-efa86eeb137a/volumes"
Jul 12 18:37:34.871489 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:37:34.871435    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:37:36.463704 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:37:36.463668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:11.750139 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:11.750110    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:21.182642 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:21.182608    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:21.282174 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:21.282143    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:21.395029 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:21.395000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:27.112101 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:27.112065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:27.172397 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:27.172366    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:27.239449 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:27.239418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:27.339235 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:27.339200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:27.455940 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:27.455855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:33.162208 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:33.162174    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:33.219097 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:33.219065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:54.179632 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:54.179601    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:38:54.231675 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:38:54.231648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:40:01.368518 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:40:01.368484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:40:01.420549 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:40:01.420520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:40:17.542507 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:40:17.542480    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:40:17.543150 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:40:17.543131    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:45:01.057096 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:45:01.057055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:45:02.584516 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:45:02.584480    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:45:17.563125 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:45:17.563096    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:45:17.566007 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:45:17.565980    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:45:22.397492 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:45:22.397447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:47:42.797109 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:47:42.797036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:49:38.462129 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:49:38.462037    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:49:47.942990 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:49:47.942959    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:49:56.191347 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:49:56.191313    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:01.262288 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:01.262221    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:04.024716 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:04.024679    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:05.370392 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:05.370364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:11.080325 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.080286    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:11.182371 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.182326    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:11.657039 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.656994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:11.740712 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.740678    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"]
Jul 12 18:50:11.741069 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.741053    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd4879b9-fb77-486e-8b09-efa86eeb137a" containerName="authorino"
Jul 12 18:50:11.741146 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.741072    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd4879b9-fb77-486e-8b09-efa86eeb137a" containerName="authorino"
Jul 12 18:50:11.741202 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.741146    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd4879b9-fb77-486e-8b09-efa86eeb137a" containerName="authorino"
Jul 12 18:50:11.743767 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.743739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:11.744038 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.744020    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:11.747835 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.747813    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2953509d-serving-cert\""
Jul 12 18:50:11.755388 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.755366    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"]
Jul 12 18:50:11.764444 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.764417    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd58a10d-eb34-4bf8-a316-39273b297bc6-maas-api-tls\") pod \"maas-api-e2e-ait-2953509d-846f5df457-rzmc5\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:11.764574 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.764540    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd58a10d-eb34-4bf8-a316-39273b297bc6-openshift-service-ca\") pod \"maas-api-e2e-ait-2953509d-846f5df457-rzmc5\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:11.764619 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.764605    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hlrq\" (UniqueName: \"kubernetes.io/projected/cd58a10d-eb34-4bf8-a316-39273b297bc6-kube-api-access-2hlrq\") pod \"maas-api-e2e-ait-2953509d-846f5df457-rzmc5\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:11.816537 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.816500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:11.865760 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.865726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd58a10d-eb34-4bf8-a316-39273b297bc6-openshift-service-ca\") pod \"maas-api-e2e-ait-2953509d-846f5df457-rzmc5\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:11.865912 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.865780    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hlrq\" (UniqueName: \"kubernetes.io/projected/cd58a10d-eb34-4bf8-a316-39273b297bc6-kube-api-access-2hlrq\") pod \"maas-api-e2e-ait-2953509d-846f5df457-rzmc5\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:11.865912 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.865825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd58a10d-eb34-4bf8-a316-39273b297bc6-maas-api-tls\") pod \"maas-api-e2e-ait-2953509d-846f5df457-rzmc5\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:11.865993 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:50:11.865948    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-serving-cert: secret "maas-api-e2e-ait-2953509d-serving-cert" not found
Jul 12 18:50:11.866031 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:50:11.866016    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd58a10d-eb34-4bf8-a316-39273b297bc6-maas-api-tls podName:cd58a10d-eb34-4bf8-a316-39273b297bc6 nodeName:}" failed. No retries permitted until 2026-07-12 18:50:12.365995579 +0000 UTC m=+1495.249345848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cd58a10d-eb34-4bf8-a316-39273b297bc6-maas-api-tls") pod "maas-api-e2e-ait-2953509d-846f5df457-rzmc5" (UID: "cd58a10d-eb34-4bf8-a316-39273b297bc6") : secret "maas-api-e2e-ait-2953509d-serving-cert" not found
Jul 12 18:50:11.866555 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.866534    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd58a10d-eb34-4bf8-a316-39273b297bc6-openshift-service-ca\") pod \"maas-api-e2e-ait-2953509d-846f5df457-rzmc5\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:11.879286 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:11.879260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hlrq\" (UniqueName: \"kubernetes.io/projected/cd58a10d-eb34-4bf8-a316-39273b297bc6-kube-api-access-2hlrq\") pod \"maas-api-e2e-ait-2953509d-846f5df457-rzmc5\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:12.015751 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:12.015715    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:12.241663 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:12.241624    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:12.350798 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:12.350716    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:12.369673 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:12.369640    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd58a10d-eb34-4bf8-a316-39273b297bc6-maas-api-tls\") pod \"maas-api-e2e-ait-2953509d-846f5df457-rzmc5\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:12.372030 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:12.372009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd58a10d-eb34-4bf8-a316-39273b297bc6-maas-api-tls\") pod \"maas-api-e2e-ait-2953509d-846f5df457-rzmc5\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:12.656001 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:12.655905    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:12.787264 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:12.787238    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"]
Jul 12 18:50:12.789981 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:50:12.789952    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd58a10d_eb34_4bf8_a316_39273b297bc6.slice/crio-239f76a76d19a5bfbb305f8d36d30d33a062e2b96ccacac127d36ee7829af277 WatchSource:0}: Error finding container 239f76a76d19a5bfbb305f8d36d30d33a062e2b96ccacac127d36ee7829af277: Status 404 returned error can't find the container with id 239f76a76d19a5bfbb305f8d36d30d33a062e2b96ccacac127d36ee7829af277
Jul 12 18:50:12.791598 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:12.791582    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 18:50:13.509983 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:13.509947    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5" event={"ID":"cd58a10d-eb34-4bf8-a316-39273b297bc6","Type":"ContainerStarted","Data":"4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88"}
Jul 12 18:50:13.510385 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:13.510034    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5" event={"ID":"cd58a10d-eb34-4bf8-a316-39273b297bc6","Type":"ContainerStarted","Data":"239f76a76d19a5bfbb305f8d36d30d33a062e2b96ccacac127d36ee7829af277"}
Jul 12 18:50:13.510385 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:13.510064    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:13.525673 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:13.525624    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5" podStartSLOduration=2.025787969 podStartE2EDuration="2.525609072s" totalImagesPullingTime="499.821103ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:50:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:50:13.525199577 +0000 UTC m=+1496.408549851" watchObservedRunningTime="2026-07-12 18:50:13.525609072 +0000 UTC m=+1496.408959347"
Jul 12 18:50:14.060957 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:14.060926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:16.754506 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:16.754476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:16.939490 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:16.939440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:17.039396 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.039302    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:17.101992 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.101959    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"]
Jul 12 18:50:17.102205 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.102175    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5" podUID="cd58a10d-eb34-4bf8-a316-39273b297bc6" containerName="maas-api" containerID="cri-o://4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88" gracePeriod=30
Jul 12 18:50:17.140521 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.140484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:17.346319 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.346294    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:17.413998 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.413958    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd58a10d-eb34-4bf8-a316-39273b297bc6-openshift-service-ca\") pod \"cd58a10d-eb34-4bf8-a316-39273b297bc6\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") "
Jul 12 18:50:17.413998 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.414000    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd58a10d-eb34-4bf8-a316-39273b297bc6-maas-api-tls\") pod \"cd58a10d-eb34-4bf8-a316-39273b297bc6\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") "
Jul 12 18:50:17.414211 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.414036    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hlrq\" (UniqueName: \"kubernetes.io/projected/cd58a10d-eb34-4bf8-a316-39273b297bc6-kube-api-access-2hlrq\") pod \"cd58a10d-eb34-4bf8-a316-39273b297bc6\" (UID: \"cd58a10d-eb34-4bf8-a316-39273b297bc6\") "
Jul 12 18:50:17.414333 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.414311    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd58a10d-eb34-4bf8-a316-39273b297bc6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cd58a10d-eb34-4bf8-a316-39273b297bc6" (UID: "cd58a10d-eb34-4bf8-a316-39273b297bc6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:50:17.416189 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.416157    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd58a10d-eb34-4bf8-a316-39273b297bc6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cd58a10d-eb34-4bf8-a316-39273b297bc6" (UID: "cd58a10d-eb34-4bf8-a316-39273b297bc6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:50:17.416189 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.416176    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd58a10d-eb34-4bf8-a316-39273b297bc6-kube-api-access-2hlrq" (OuterVolumeSpecName: "kube-api-access-2hlrq") pod "cd58a10d-eb34-4bf8-a316-39273b297bc6" (UID: "cd58a10d-eb34-4bf8-a316-39273b297bc6"). InnerVolumeSpecName "kube-api-access-2hlrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:50:17.514650 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.514624    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd58a10d-eb34-4bf8-a316-39273b297bc6-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:50:17.514650 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.514649    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd58a10d-eb34-4bf8-a316-39273b297bc6-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:50:17.514650 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.514658    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hlrq\" (UniqueName: \"kubernetes.io/projected/cd58a10d-eb34-4bf8-a316-39273b297bc6-kube-api-access-2hlrq\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:50:17.528093 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.528065    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd58a10d-eb34-4bf8-a316-39273b297bc6" containerID="4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88" exitCode=0
Jul 12 18:50:17.528236 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.528112    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"
Jul 12 18:50:17.528236 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.528150    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5" event={"ID":"cd58a10d-eb34-4bf8-a316-39273b297bc6","Type":"ContainerDied","Data":"4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88"}
Jul 12 18:50:17.528236 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.528212    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5" event={"ID":"cd58a10d-eb34-4bf8-a316-39273b297bc6","Type":"ContainerDied","Data":"239f76a76d19a5bfbb305f8d36d30d33a062e2b96ccacac127d36ee7829af277"}
Jul 12 18:50:17.528236 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.528235    2562 scope.go:117] "RemoveContainer" containerID="4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88"
Jul 12 18:50:17.536818 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.536802    2562 scope.go:117] "RemoveContainer" containerID="4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88"
Jul 12 18:50:17.537084 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:50:17.537065    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88\": container with ID starting with 4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88 not found: ID does not exist" containerID="4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88"
Jul 12 18:50:17.537196 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.537090    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88"} err="failed to get container status \"4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88\": rpc error: code = NotFound desc = could not find container \"4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88\": container with ID starting with 4fafc6af8e03ff1744e31a2caaf590f13917eb344655ece5f5a3e2b0cebc7a88 not found: ID does not exist"
Jul 12 18:50:17.549857 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.549835    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"]
Jul 12 18:50:17.554290 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.554270    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2953509d-846f5df457-rzmc5"]
Jul 12 18:50:17.585281 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.585208    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:50:17.588565 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.588534    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:50:17.604824 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:17.604797    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd58a10d-eb34-4bf8-a316-39273b297bc6" path="/var/lib/kubelet/pods/cd58a10d-eb34-4bf8-a316-39273b297bc6/volumes"
Jul 12 18:50:20.096220 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:20.096189    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:22.633372 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:22.633329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:22.698553 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:22.698521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:22.834963 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:22.834933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:22.889383 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:22.889286    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:23.926797 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:23.926765    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:24.010893 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.010855    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"]
Jul 12 18:50:24.011277 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.011261    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd58a10d-eb34-4bf8-a316-39273b297bc6" containerName="maas-api"
Jul 12 18:50:24.011277 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.011278    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd58a10d-eb34-4bf8-a316-39273b297bc6" containerName="maas-api"
Jul 12 18:50:24.011438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.011367    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd58a10d-eb34-4bf8-a316-39273b297bc6" containerName="maas-api"
Jul 12 18:50:24.013161 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.013135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:24.016056 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.016034    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.018048 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.018030    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-73e47abc-serving-cert\""
Jul 12 18:50:24.025634 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.025610    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"]
Jul 12 18:50:24.071415 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.071384    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3131be4-7409-4624-827b-a65145423ab2-maas-api-tls\") pod \"maas-api-e2e-ait-73e47abc-846b766886-b95k5\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.071569 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.071451    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bklfj\" (UniqueName: \"kubernetes.io/projected/e3131be4-7409-4624-827b-a65145423ab2-kube-api-access-bklfj\") pod \"maas-api-e2e-ait-73e47abc-846b766886-b95k5\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.071569 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.071506    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3131be4-7409-4624-827b-a65145423ab2-openshift-service-ca\") pod \"maas-api-e2e-ait-73e47abc-846b766886-b95k5\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.088296 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.088265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:24.171956 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.171925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3131be4-7409-4624-827b-a65145423ab2-openshift-service-ca\") pod \"maas-api-e2e-ait-73e47abc-846b766886-b95k5\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.172119 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.171984    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3131be4-7409-4624-827b-a65145423ab2-maas-api-tls\") pod \"maas-api-e2e-ait-73e47abc-846b766886-b95k5\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.172119 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.172028    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bklfj\" (UniqueName: \"kubernetes.io/projected/e3131be4-7409-4624-827b-a65145423ab2-kube-api-access-bklfj\") pod \"maas-api-e2e-ait-73e47abc-846b766886-b95k5\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.172571 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.172547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3131be4-7409-4624-827b-a65145423ab2-openshift-service-ca\") pod \"maas-api-e2e-ait-73e47abc-846b766886-b95k5\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.174320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.174300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3131be4-7409-4624-827b-a65145423ab2-maas-api-tls\") pod \"maas-api-e2e-ait-73e47abc-846b766886-b95k5\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.179792 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.179733    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bklfj\" (UniqueName: \"kubernetes.io/projected/e3131be4-7409-4624-827b-a65145423ab2-kube-api-access-bklfj\") pod \"maas-api-e2e-ait-73e47abc-846b766886-b95k5\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.188036 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.188018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:24.328235 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.328204    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:24.465882 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.465750    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"]
Jul 12 18:50:24.468533 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:50:24.468505    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3131be4_7409_4624_827b_a65145423ab2.slice/crio-c24284895737dcd8e87f7780c0f0da5bd3fbdd0152ab04ae4d95c3db8d65c8f6 WatchSource:0}: Error finding container c24284895737dcd8e87f7780c0f0da5bd3fbdd0152ab04ae4d95c3db8d65c8f6: Status 404 returned error can't find the container with id c24284895737dcd8e87f7780c0f0da5bd3fbdd0152ab04ae4d95c3db8d65c8f6
Jul 12 18:50:24.556630 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.556595    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5" event={"ID":"e3131be4-7409-4624-827b-a65145423ab2","Type":"ContainerStarted","Data":"c24284895737dcd8e87f7780c0f0da5bd3fbdd0152ab04ae4d95c3db8d65c8f6"}
Jul 12 18:50:24.604469 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.604424    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:24.703601 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:24.703572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:25.561964 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:25.561930    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5" event={"ID":"e3131be4-7409-4624-827b-a65145423ab2","Type":"ContainerStarted","Data":"8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a"}
Jul 12 18:50:25.562353 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:25.562059    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:25.579066 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:25.579022    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5" podStartSLOduration=2.2670686509999998 podStartE2EDuration="2.579005266s" totalImagesPullingTime="311.936615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:50:25.576838613 +0000 UTC m=+1508.460188887" watchObservedRunningTime="2026-07-12 18:50:25.579005266 +0000 UTC m=+1508.462355539"
Jul 12 18:50:25.670123 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:25.670086    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:29.023011 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.022975    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:29.509536 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.509505    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"]
Jul 12 18:50:29.509767 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.509740    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5" podUID="e3131be4-7409-4624-827b-a65145423ab2" containerName="maas-api" containerID="cri-o://8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a" gracePeriod=30
Jul 12 18:50:29.515619 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.515595    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:29.532722 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.532700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:29.753659 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.753630    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:29.827933 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.827848    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3131be4-7409-4624-827b-a65145423ab2-openshift-service-ca\") pod \"e3131be4-7409-4624-827b-a65145423ab2\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") "
Jul 12 18:50:29.827933 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.827920    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bklfj\" (UniqueName: \"kubernetes.io/projected/e3131be4-7409-4624-827b-a65145423ab2-kube-api-access-bklfj\") pod \"e3131be4-7409-4624-827b-a65145423ab2\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") "
Jul 12 18:50:29.828148 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.827985    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3131be4-7409-4624-827b-a65145423ab2-maas-api-tls\") pod \"e3131be4-7409-4624-827b-a65145423ab2\" (UID: \"e3131be4-7409-4624-827b-a65145423ab2\") "
Jul 12 18:50:29.828281 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.828250    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3131be4-7409-4624-827b-a65145423ab2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e3131be4-7409-4624-827b-a65145423ab2" (UID: "e3131be4-7409-4624-827b-a65145423ab2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:50:29.830033 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.830009    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3131be4-7409-4624-827b-a65145423ab2-kube-api-access-bklfj" (OuterVolumeSpecName: "kube-api-access-bklfj") pod "e3131be4-7409-4624-827b-a65145423ab2" (UID: "e3131be4-7409-4624-827b-a65145423ab2"). InnerVolumeSpecName "kube-api-access-bklfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:50:29.830136 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.830045    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3131be4-7409-4624-827b-a65145423ab2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e3131be4-7409-4624-827b-a65145423ab2" (UID: "e3131be4-7409-4624-827b-a65145423ab2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:50:29.929127 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.929095    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3131be4-7409-4624-827b-a65145423ab2-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:50:29.929127 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.929121    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3131be4-7409-4624-827b-a65145423ab2-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:50:29.929127 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:29.929133    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bklfj\" (UniqueName: \"kubernetes.io/projected/e3131be4-7409-4624-827b-a65145423ab2-kube-api-access-bklfj\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:50:30.583464 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:30.583432    2562 generic.go:356] "Generic (PLEG): container finished" podID="e3131be4-7409-4624-827b-a65145423ab2" containerID="8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a" exitCode=0
Jul 12 18:50:30.583874 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:30.583492    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"
Jul 12 18:50:30.583874 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:30.583517    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5" event={"ID":"e3131be4-7409-4624-827b-a65145423ab2","Type":"ContainerDied","Data":"8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a"}
Jul 12 18:50:30.583874 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:30.583567    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5" event={"ID":"e3131be4-7409-4624-827b-a65145423ab2","Type":"ContainerDied","Data":"c24284895737dcd8e87f7780c0f0da5bd3fbdd0152ab04ae4d95c3db8d65c8f6"}
Jul 12 18:50:30.583874 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:30.583598    2562 scope.go:117] "RemoveContainer" containerID="8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a"
Jul 12 18:50:30.592314 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:30.592300    2562 scope.go:117] "RemoveContainer" containerID="8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a"
Jul 12 18:50:30.592597 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:50:30.592578    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a\": container with ID starting with 8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a not found: ID does not exist" containerID="8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a"
Jul 12 18:50:30.592644 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:30.592605    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a"} err="failed to get container status \"8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a\": rpc error: code = NotFound desc = could not find container \"8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a\": container with ID starting with 8a794a1d6bb40059408b17246b1c978100d41d7905a4ab3c58571124f026112a not found: ID does not exist"
Jul 12 18:50:30.602786 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:30.602766    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"]
Jul 12 18:50:30.605246 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:30.605226    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e47abc-846b766886-b95k5"]
Jul 12 18:50:31.602112 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:31.602083    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3131be4-7409-4624-827b-a65145423ab2" path="/var/lib/kubelet/pods/e3131be4-7409-4624-827b-a65145423ab2/volumes"
Jul 12 18:50:35.085127 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:35.085094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:35.184588 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:35.184556    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:38.775246 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:38.775210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:41.113089 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:41.113055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:41.171941 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:41.171911    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:41.323448 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:41.323421    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:41.435639 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:41.435555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:42.513859 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.513826    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:42.613220 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.613187    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:42.616705 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.616677    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"]
Jul 12 18:50:42.617076 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.617061    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3131be4-7409-4624-827b-a65145423ab2" containerName="maas-api"
Jul 12 18:50:42.617117 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.617081    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3131be4-7409-4624-827b-a65145423ab2" containerName="maas-api"
Jul 12 18:50:42.617178 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.617168    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3131be4-7409-4624-827b-a65145423ab2" containerName="maas-api"
Jul 12 18:50:42.621647 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.621629    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:42.623872 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.623851    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-1ad2a327-serving-cert\""
Jul 12 18:50:42.630732 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.630713    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"]
Jul 12 18:50:42.675431 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.675400    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:42.745306 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.745273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7931776c-3b1f-4cb0-b726-86c47c8df760-openshift-service-ca\") pod \"maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:42.745438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.745346    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7931776c-3b1f-4cb0-b726-86c47c8df760-maas-api-tls\") pod \"maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:42.745438 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.745411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhf98\" (UniqueName: \"kubernetes.io/projected/7931776c-3b1f-4cb0-b726-86c47c8df760-kube-api-access-lhf98\") pod \"maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:42.826935 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.826848    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:42.846376 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.846343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7931776c-3b1f-4cb0-b726-86c47c8df760-openshift-service-ca\") pod \"maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:42.846540 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.846433    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7931776c-3b1f-4cb0-b726-86c47c8df760-maas-api-tls\") pod \"maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:42.846540 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.846502    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhf98\" (UniqueName: \"kubernetes.io/projected/7931776c-3b1f-4cb0-b726-86c47c8df760-kube-api-access-lhf98\") pod \"maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:42.846965 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.846946    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7931776c-3b1f-4cb0-b726-86c47c8df760-openshift-service-ca\") pod \"maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:42.848900 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.848872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7931776c-3b1f-4cb0-b726-86c47c8df760-maas-api-tls\") pod \"maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:42.855855 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.855831    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhf98\" (UniqueName: \"kubernetes.io/projected/7931776c-3b1f-4cb0-b726-86c47c8df760-kube-api-access-lhf98\") pod \"maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:42.933794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:42.933760    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:43.070279 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:43.070241    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"]
Jul 12 18:50:43.073668 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:43.073640    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:43.074580 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:50:43.074558    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7931776c_3b1f_4cb0_b726_86c47c8df760.slice/crio-093b352008576ce5c1ae7352503b56d95b6c34545afb26cdf0feada4c6e4bcb9 WatchSource:0}: Error finding container 093b352008576ce5c1ae7352503b56d95b6c34545afb26cdf0feada4c6e4bcb9: Status 404 returned error can't find the container with id 093b352008576ce5c1ae7352503b56d95b6c34545afb26cdf0feada4c6e4bcb9
Jul 12 18:50:43.176430 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:43.176384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:43.356543 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:43.356445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:43.644327 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:43.644238    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249" event={"ID":"7931776c-3b1f-4cb0-b726-86c47c8df760","Type":"ContainerStarted","Data":"1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe"}
Jul 12 18:50:43.644327 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:43.644310    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249" event={"ID":"7931776c-3b1f-4cb0-b726-86c47c8df760","Type":"ContainerStarted","Data":"093b352008576ce5c1ae7352503b56d95b6c34545afb26cdf0feada4c6e4bcb9"}
Jul 12 18:50:43.644755 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:43.644345    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:43.662387 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:43.662331    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249" podStartSLOduration=1.366384887 podStartE2EDuration="1.662311454s" totalImagesPullingTime="295.926567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:50:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:50:43.659989927 +0000 UTC m=+1526.543340197" watchObservedRunningTime="2026-07-12 18:50:43.662311454 +0000 UTC m=+1526.545661733"
Jul 12 18:50:44.373331 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:44.373299    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:46.851476 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:46.851428    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:47.075849 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.075816    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:47.176365 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.176281    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:47.203559 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.203530    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"]
Jul 12 18:50:47.203777 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.203749    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249" podUID="7931776c-3b1f-4cb0-b726-86c47c8df760" containerName="maas-api" containerID="cri-o://1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe" gracePeriod=30
Jul 12 18:50:47.276574 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.276529    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:47.435043 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.434993    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:47.484606 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.484578    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7931776c-3b1f-4cb0-b726-86c47c8df760-openshift-service-ca\") pod \"7931776c-3b1f-4cb0-b726-86c47c8df760\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") "
Jul 12 18:50:47.484761 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.484611    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7931776c-3b1f-4cb0-b726-86c47c8df760-maas-api-tls\") pod \"7931776c-3b1f-4cb0-b726-86c47c8df760\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") "
Jul 12 18:50:47.484761 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.484667    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhf98\" (UniqueName: \"kubernetes.io/projected/7931776c-3b1f-4cb0-b726-86c47c8df760-kube-api-access-lhf98\") pod \"7931776c-3b1f-4cb0-b726-86c47c8df760\" (UID: \"7931776c-3b1f-4cb0-b726-86c47c8df760\") "
Jul 12 18:50:47.484928 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.484906    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7931776c-3b1f-4cb0-b726-86c47c8df760-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7931776c-3b1f-4cb0-b726-86c47c8df760" (UID: "7931776c-3b1f-4cb0-b726-86c47c8df760"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:50:47.486740 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.486715    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7931776c-3b1f-4cb0-b726-86c47c8df760-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7931776c-3b1f-4cb0-b726-86c47c8df760" (UID: "7931776c-3b1f-4cb0-b726-86c47c8df760"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:50:47.486844 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.486745    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7931776c-3b1f-4cb0-b726-86c47c8df760-kube-api-access-lhf98" (OuterVolumeSpecName: "kube-api-access-lhf98") pod "7931776c-3b1f-4cb0-b726-86c47c8df760" (UID: "7931776c-3b1f-4cb0-b726-86c47c8df760"). InnerVolumeSpecName "kube-api-access-lhf98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:50:47.585185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.585150    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7931776c-3b1f-4cb0-b726-86c47c8df760-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:50:47.585185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.585177    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7931776c-3b1f-4cb0-b726-86c47c8df760-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:50:47.585185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.585187    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhf98\" (UniqueName: \"kubernetes.io/projected/7931776c-3b1f-4cb0-b726-86c47c8df760-kube-api-access-lhf98\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:50:47.662197 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.662164    2562 generic.go:356] "Generic (PLEG): container finished" podID="7931776c-3b1f-4cb0-b726-86c47c8df760" containerID="1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe" exitCode=0
Jul 12 18:50:47.662384 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.662214    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"
Jul 12 18:50:47.662384 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.662245    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249" event={"ID":"7931776c-3b1f-4cb0-b726-86c47c8df760","Type":"ContainerDied","Data":"1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe"}
Jul 12 18:50:47.662384 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.662308    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249" event={"ID":"7931776c-3b1f-4cb0-b726-86c47c8df760","Type":"ContainerDied","Data":"093b352008576ce5c1ae7352503b56d95b6c34545afb26cdf0feada4c6e4bcb9"}
Jul 12 18:50:47.662384 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.662330    2562 scope.go:117] "RemoveContainer" containerID="1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe"
Jul 12 18:50:47.670244 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.670226    2562 scope.go:117] "RemoveContainer" containerID="1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe"
Jul 12 18:50:47.670546 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:50:47.670526    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe\": container with ID starting with 1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe not found: ID does not exist" containerID="1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe"
Jul 12 18:50:47.670595 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.670558    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe"} err="failed to get container status \"1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe\": rpc error: code = NotFound desc = could not find container \"1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe\": container with ID starting with 1a4a87f1f0179ba16bafca60d3f7a5453ab347ede08471897a7bb8d3a65cedfe not found: ID does not exist"
Jul 12 18:50:47.679531 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.679510    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"]
Jul 12 18:50:47.683799 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:47.683780    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ad2a327-5d7bcc6684-52249"]
Jul 12 18:50:48.549094 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:48.549068    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:49.601836 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:49.601800    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7931776c-3b1f-4cb0-b726-86c47c8df760" path="/var/lib/kubelet/pods/7931776c-3b1f-4cb0-b726-86c47c8df760/volumes"
Jul 12 18:50:53.010517 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:53.010478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:53.111086 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:53.111055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:53.473853 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:53.473821    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:53.570522 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:53.570486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:53.633567 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:53.633536    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:50:55.008170 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:50:55.008136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:00.875745 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:00.875715    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:00.991126 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:00.991100    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:01.124808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:01.124773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:01.238211 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:01.238182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:01.575690 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:01.575599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:27.534173 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:27.534137    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:33.118560 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:33.118517    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:33.223577 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:33.223541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:33.321889 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:33.321856    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:49.181985 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:49.181953    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:54.966286 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:54.966252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:55.027109 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:55.027086    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:55.102101 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:55.102068    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:51:55.199668 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:51:55.199639    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:05.686057 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:05.686024    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:06.469656 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:06.469628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:12.282373 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:12.282284    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:12.384907 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:12.384873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:12.482018 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:12.481988    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:12.583570 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:12.583483    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:13.348049 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:13.348011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:13.446758 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:13.446728    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:13.542958 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:13.542927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:15.143036 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:15.143005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:20.986991 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:20.986961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:21.087248 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:21.087213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:21.646791 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:21.646760    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:27.377718 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:27.377678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:27.445353 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:27.445322    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:27.615913 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:27.615881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:27.917505 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:27.917478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:28.242818 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:28.242789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:31.021794 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.021760    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:31.126471 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.126422    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"]
Jul 12 18:52:31.126722 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.126670    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:31.126853 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.126835    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7931776c-3b1f-4cb0-b726-86c47c8df760" containerName="maas-api"
Jul 12 18:52:31.126853 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.126852    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7931776c-3b1f-4cb0-b726-86c47c8df760" containerName="maas-api"
Jul 12 18:52:31.126966 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.126916    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7931776c-3b1f-4cb0-b726-86c47c8df760" containerName="maas-api"
Jul 12 18:52:31.129831 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.129812    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.132136 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.132108    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1784bded-serving-cert\""
Jul 12 18:52:31.138987 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.138967    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"]
Jul 12 18:52:31.149546 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.149522    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca702a9f-b7bc-4b7e-a339-1d41046377c4-openshift-service-ca\") pod \"maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.149660 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.149586    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca702a9f-b7bc-4b7e-a339-1d41046377c4-maas-api-tls\") pod \"maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.149660 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.149636    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czcpr\" (UniqueName: \"kubernetes.io/projected/ca702a9f-b7bc-4b7e-a339-1d41046377c4-kube-api-access-czcpr\") pod \"maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.222336 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.222303    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:31.250306 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.250274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca702a9f-b7bc-4b7e-a339-1d41046377c4-openshift-service-ca\") pod \"maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.250452 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.250313    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca702a9f-b7bc-4b7e-a339-1d41046377c4-maas-api-tls\") pod \"maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.250452 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.250337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czcpr\" (UniqueName: \"kubernetes.io/projected/ca702a9f-b7bc-4b7e-a339-1d41046377c4-kube-api-access-czcpr\") pod \"maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.250963 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.250942    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca702a9f-b7bc-4b7e-a339-1d41046377c4-openshift-service-ca\") pod \"maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.252708 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.252687    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca702a9f-b7bc-4b7e-a339-1d41046377c4-maas-api-tls\") pod \"maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.259237 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.259215    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czcpr\" (UniqueName: \"kubernetes.io/projected/ca702a9f-b7bc-4b7e-a339-1d41046377c4-kube-api-access-czcpr\") pod \"maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.322857 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.322773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:31.440612 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.440580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:31.570905 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.570717    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"]
Jul 12 18:52:31.573145 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:52:31.573078    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca702a9f_b7bc_4b7e_a339_1d41046377c4.slice/crio-f159f9cd84ab1daeb316c7b5fb465be4cc53889e8fbd5e3089015ab44105e037 WatchSource:0}: Error finding container f159f9cd84ab1daeb316c7b5fb465be4cc53889e8fbd5e3089015ab44105e037: Status 404 returned error can't find the container with id f159f9cd84ab1daeb316c7b5fb465be4cc53889e8fbd5e3089015ab44105e037
Jul 12 18:52:31.577714 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.577689    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:31.676945 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:31.676918    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:32.076431 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:32.076398    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c" event={"ID":"ca702a9f-b7bc-4b7e-a339-1d41046377c4","Type":"ContainerStarted","Data":"5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044"}
Jul 12 18:52:32.076827 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:32.076447    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c" event={"ID":"ca702a9f-b7bc-4b7e-a339-1d41046377c4","Type":"ContainerStarted","Data":"f159f9cd84ab1daeb316c7b5fb465be4cc53889e8fbd5e3089015ab44105e037"}
Jul 12 18:52:32.076827 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:32.076531    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:32.094273 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:32.094221    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c" podStartSLOduration=0.715386549 podStartE2EDuration="1.094207168s" totalImagesPullingTime="378.820619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:52:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:52:32.092858762 +0000 UTC m=+1634.976209037" watchObservedRunningTime="2026-07-12 18:52:32.094207168 +0000 UTC m=+1634.977557441"
Jul 12 18:52:35.516595 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:35.516562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:36.157388 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:36.157355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:36.257381 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:36.257349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:36.582742 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:36.582711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:38.086128 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:38.086102    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:42.264685 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:42.264652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:42.363577 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:42.363542    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:42.646913 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:42.646830    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:43.039929 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.039897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:43.149568 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.149538    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"]
Jul 12 18:52:43.149808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.149775    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c" podUID="ca702a9f-b7bc-4b7e-a339-1d41046377c4" containerName="maas-api" containerID="cri-o://5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044" gracePeriod=30
Jul 12 18:52:43.172999 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.172977    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:43.393614 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.393593    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:43.461394 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.461355    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca702a9f-b7bc-4b7e-a339-1d41046377c4-maas-api-tls\") pod \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") "
Jul 12 18:52:43.461620 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.461445    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca702a9f-b7bc-4b7e-a339-1d41046377c4-openshift-service-ca\") pod \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") "
Jul 12 18:52:43.461620 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.461553    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czcpr\" (UniqueName: \"kubernetes.io/projected/ca702a9f-b7bc-4b7e-a339-1d41046377c4-kube-api-access-czcpr\") pod \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\" (UID: \"ca702a9f-b7bc-4b7e-a339-1d41046377c4\") "
Jul 12 18:52:43.461811 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.461790    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca702a9f-b7bc-4b7e-a339-1d41046377c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca702a9f-b7bc-4b7e-a339-1d41046377c4" (UID: "ca702a9f-b7bc-4b7e-a339-1d41046377c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:52:43.463448 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.463423    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca702a9f-b7bc-4b7e-a339-1d41046377c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca702a9f-b7bc-4b7e-a339-1d41046377c4" (UID: "ca702a9f-b7bc-4b7e-a339-1d41046377c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:52:43.463585 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.463563    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca702a9f-b7bc-4b7e-a339-1d41046377c4-kube-api-access-czcpr" (OuterVolumeSpecName: "kube-api-access-czcpr") pod "ca702a9f-b7bc-4b7e-a339-1d41046377c4" (UID: "ca702a9f-b7bc-4b7e-a339-1d41046377c4"). InnerVolumeSpecName "kube-api-access-czcpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:52:43.562410 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.562374    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca702a9f-b7bc-4b7e-a339-1d41046377c4-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:52:43.562410 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.562405    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca702a9f-b7bc-4b7e-a339-1d41046377c4-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:52:43.562410 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:43.562419    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czcpr\" (UniqueName: \"kubernetes.io/projected/ca702a9f-b7bc-4b7e-a339-1d41046377c4-kube-api-access-czcpr\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:52:44.124214 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:44.124181    2562 generic.go:356] "Generic (PLEG): container finished" podID="ca702a9f-b7bc-4b7e-a339-1d41046377c4" containerID="5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044" exitCode=0
Jul 12 18:52:44.124389 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:44.124243    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"
Jul 12 18:52:44.124389 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:44.124254    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c" event={"ID":"ca702a9f-b7bc-4b7e-a339-1d41046377c4","Type":"ContainerDied","Data":"5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044"}
Jul 12 18:52:44.124389 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:44.124296    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c" event={"ID":"ca702a9f-b7bc-4b7e-a339-1d41046377c4","Type":"ContainerDied","Data":"f159f9cd84ab1daeb316c7b5fb465be4cc53889e8fbd5e3089015ab44105e037"}
Jul 12 18:52:44.124389 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:44.124315    2562 scope.go:117] "RemoveContainer" containerID="5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044"
Jul 12 18:52:44.132210 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:44.132192    2562 scope.go:117] "RemoveContainer" containerID="5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044"
Jul 12 18:52:44.132469 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:52:44.132430    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044\": container with ID starting with 5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044 not found: ID does not exist" containerID="5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044"
Jul 12 18:52:44.132529 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:44.132489    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044"} err="failed to get container status \"5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044\": rpc error: code = NotFound desc = could not find container \"5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044\": container with ID starting with 5f27fb7bafe85da5dbb0d1f809a8e896a4edd040055efe593bf3dc8fdc742044 not found: ID does not exist"
Jul 12 18:52:44.138982 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:44.138960    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"]
Jul 12 18:52:44.144896 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:44.144876    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1784bded-778c8ddfb6-xkr5c"]
Jul 12 18:52:45.601878 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:45.601846    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca702a9f-b7bc-4b7e-a339-1d41046377c4" path="/var/lib/kubelet/pods/ca702a9f-b7bc-4b7e-a339-1d41046377c4/volumes"
Jul 12 18:52:48.736626 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:48.736599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:48.823195 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:48.823158    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:48.987271 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:48.987193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:49.328721 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:49.328642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:49.426982 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:49.426954    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:49.530589 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:49.530560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:55.111386 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:55.111353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:55.216266 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:55.216233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:55.311132 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:55.311099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:52:55.455986 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:52:55.455907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:00.632185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:00.632156    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:00.733030 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:00.732995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:01.227538 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:01.227505    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:01.327531 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:01.327502    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:07.905115 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:07.905075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:08.187509 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:08.187414    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:08.463509 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:08.463415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:08.562047 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:08.562009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:11.226185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.226149    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:11.325672 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.325636    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:11.508644 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.508571    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"]
Jul 12 18:53:11.508887 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.508874    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca702a9f-b7bc-4b7e-a339-1d41046377c4" containerName="maas-api"
Jul 12 18:53:11.508887 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.508887    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca702a9f-b7bc-4b7e-a339-1d41046377c4" containerName="maas-api"
Jul 12 18:53:11.509017 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.508945    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca702a9f-b7bc-4b7e-a339-1d41046377c4" containerName="maas-api"
Jul 12 18:53:11.511912 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.511893    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.514225 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.514195    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-443e5c6f-serving-cert\""
Jul 12 18:53:11.523252 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.523232    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"]
Jul 12 18:53:11.546923 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.546899    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:11.604360 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.604331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/348fbf1d-2f31-4e07-9d44-067716da3d6e-openshift-service-ca\") pod \"maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.604532 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.604365    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98bd5\" (UniqueName: \"kubernetes.io/projected/348fbf1d-2f31-4e07-9d44-067716da3d6e-kube-api-access-98bd5\") pod \"maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.604532 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.604446    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/348fbf1d-2f31-4e07-9d44-067716da3d6e-maas-api-tls\") pod \"maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.705760 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.705720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/348fbf1d-2f31-4e07-9d44-067716da3d6e-openshift-service-ca\") pod \"maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.705760 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.705760    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98bd5\" (UniqueName: \"kubernetes.io/projected/348fbf1d-2f31-4e07-9d44-067716da3d6e-kube-api-access-98bd5\") pod \"maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.706040 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.705796    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/348fbf1d-2f31-4e07-9d44-067716da3d6e-maas-api-tls\") pod \"maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.706451 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.706426    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/348fbf1d-2f31-4e07-9d44-067716da3d6e-openshift-service-ca\") pod \"maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.707764 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.707745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:11.708374 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.708357    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/348fbf1d-2f31-4e07-9d44-067716da3d6e-maas-api-tls\") pod \"maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.713229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.713208    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98bd5\" (UniqueName: \"kubernetes.io/projected/348fbf1d-2f31-4e07-9d44-067716da3d6e-kube-api-access-98bd5\") pod \"maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.822003 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.821915    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:11.958655 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.958627    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"]
Jul 12 18:53:11.961221 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:53:11.961194    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod348fbf1d_2f31_4e07_9d44_067716da3d6e.slice/crio-32b3c584b61494874a7c39640595cff8317b8f788720b73a36d6744f794093a7 WatchSource:0}: Error finding container 32b3c584b61494874a7c39640595cff8317b8f788720b73a36d6744f794093a7: Status 404 returned error can't find the container with id 32b3c584b61494874a7c39640595cff8317b8f788720b73a36d6744f794093a7
Jul 12 18:53:11.995800 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:11.995767    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:12.097316 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:12.097228    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:12.239976 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:12.239943    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z" event={"ID":"348fbf1d-2f31-4e07-9d44-067716da3d6e","Type":"ContainerStarted","Data":"32b3c584b61494874a7c39640595cff8317b8f788720b73a36d6744f794093a7"}
Jul 12 18:53:13.245746 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:13.245715    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z" event={"ID":"348fbf1d-2f31-4e07-9d44-067716da3d6e","Type":"ContainerStarted","Data":"2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851"}
Jul 12 18:53:13.246139 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:13.245828    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:13.263027 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:13.262981    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z" podStartSLOduration=1.955413479 podStartE2EDuration="2.262965688s" totalImagesPullingTime="307.552209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:53:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:53:13.261911688 +0000 UTC m=+1676.145261962" watchObservedRunningTime="2026-07-12 18:53:13.262965688 +0000 UTC m=+1676.146315961"
Jul 12 18:53:15.720047 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:15.720012    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:19.255384 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:19.255357    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:37.909503 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:37.909472    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"]
Jul 12 18:53:37.912639 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:37.912616    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:37.915542 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:37.915519    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-443e5c6f\"/\"e2e-lifecycle-model-443e5c6f-kserve-self-signed-certs\""
Jul 12 18:53:37.915822 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:37.915801    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-443e5c6f\"/\"default-dockercfg-95nzb\""
Jul 12 18:53:37.915933 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:37.915847    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-443e5c6f\"/\"kube-root-ca.crt\""
Jul 12 18:53:37.915933 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:37.915885    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-443e5c6f\"/\"openshift-service-ca.crt\""
Jul 12 18:53:37.923210 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:37.923182    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"]
Jul 12 18:53:37.941143 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:37.941119    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:38.034764 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.034734    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-home\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.034923 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.034783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lr7w\" (UniqueName: \"kubernetes.io/projected/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-kube-api-access-7lr7w\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.034923 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.034817    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-model-cache\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.034923 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.034855    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-dshm\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.034923 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.034874    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tmp-dir\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.035069 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.034929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tls-certs\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.136143 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.136058    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lr7w\" (UniqueName: \"kubernetes.io/projected/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-kube-api-access-7lr7w\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.136143 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.136112    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-model-cache\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.136143 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.136133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-dshm\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.136432 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.136159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tmp-dir\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.136432 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.136194    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tls-certs\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.136432 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.136227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-home\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.136585 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.136516    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-model-cache\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.136651 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.136598    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tmp-dir\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.136651 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.136636    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-home\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.138428 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.138402    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-dshm\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.138593 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.138578    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tls-certs\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.144098 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.144071    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lr7w\" (UniqueName: \"kubernetes.io/projected/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-kube-api-access-7lr7w\") pod \"e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") " pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.224058 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.224030    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:38.352576 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:38.352546    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"]
Jul 12 18:53:38.355407 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:53:38.355384    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d9e9766_c443_41b1_b4a8_e8f9ed9008b6.slice/crio-81f55d936f031c9824191940e1a04a3553d5bef20aa17f7947bad3888e83d3ff WatchSource:0}: Error finding container 81f55d936f031c9824191940e1a04a3553d5bef20aa17f7947bad3888e83d3ff: Status 404 returned error can't find the container with id 81f55d936f031c9824191940e1a04a3553d5bef20aa17f7947bad3888e83d3ff
Jul 12 18:53:39.352732 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:39.352657    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4" event={"ID":"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6","Type":"ContainerStarted","Data":"d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573"}
Jul 12 18:53:39.352732 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:39.352699    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4" event={"ID":"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6","Type":"ContainerStarted","Data":"81f55d936f031c9824191940e1a04a3553d5bef20aa17f7947bad3888e83d3ff"}
Jul 12 18:53:39.369229 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:39.369174    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4" podStartSLOduration=2.369156826 podStartE2EDuration="2.369156826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:53:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:53:39.368344498 +0000 UTC m=+1702.251694769" watchObservedRunningTime="2026-07-12 18:53:39.369156826 +0000 UTC m=+1702.252507100"
Jul 12 18:53:48.224493 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:48.224444    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:48.224938 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:48.224522    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:48.237112 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:48.237086    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:48.400347 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:48.400314    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:53:58.611855 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:58.611819    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:59.094717 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.094683    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"]
Jul 12 18:53:59.094935 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.094914    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z" podUID="348fbf1d-2f31-4e07-9d44-067716da3d6e" containerName="maas-api" containerID="cri-o://2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851" gracePeriod=30
Jul 12 18:53:59.118334 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.118306    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:53:59.250652 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.250607    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z" podUID="348fbf1d-2f31-4e07-9d44-067716da3d6e" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.35:8443/health\": dial tcp 10.133.0.35:8443: connect: connection refused"
Jul 12 18:53:59.332430 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.332405    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:59.416041 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.415970    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/348fbf1d-2f31-4e07-9d44-067716da3d6e-openshift-service-ca\") pod \"348fbf1d-2f31-4e07-9d44-067716da3d6e\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") "
Jul 12 18:53:59.416041 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.416026    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98bd5\" (UniqueName: \"kubernetes.io/projected/348fbf1d-2f31-4e07-9d44-067716da3d6e-kube-api-access-98bd5\") pod \"348fbf1d-2f31-4e07-9d44-067716da3d6e\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") "
Jul 12 18:53:59.416200 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.416080    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/348fbf1d-2f31-4e07-9d44-067716da3d6e-maas-api-tls\") pod \"348fbf1d-2f31-4e07-9d44-067716da3d6e\" (UID: \"348fbf1d-2f31-4e07-9d44-067716da3d6e\") "
Jul 12 18:53:59.416350 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.416323    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/348fbf1d-2f31-4e07-9d44-067716da3d6e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "348fbf1d-2f31-4e07-9d44-067716da3d6e" (UID: "348fbf1d-2f31-4e07-9d44-067716da3d6e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:53:59.418059 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.418037    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/348fbf1d-2f31-4e07-9d44-067716da3d6e-kube-api-access-98bd5" (OuterVolumeSpecName: "kube-api-access-98bd5") pod "348fbf1d-2f31-4e07-9d44-067716da3d6e" (UID: "348fbf1d-2f31-4e07-9d44-067716da3d6e"). InnerVolumeSpecName "kube-api-access-98bd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:53:59.418184 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.418087    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/348fbf1d-2f31-4e07-9d44-067716da3d6e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "348fbf1d-2f31-4e07-9d44-067716da3d6e" (UID: "348fbf1d-2f31-4e07-9d44-067716da3d6e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:53:59.435302 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.435269    2562 generic.go:356] "Generic (PLEG): container finished" podID="348fbf1d-2f31-4e07-9d44-067716da3d6e" containerID="2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851" exitCode=0
Jul 12 18:53:59.435410 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.435323    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"
Jul 12 18:53:59.435410 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.435391    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z" event={"ID":"348fbf1d-2f31-4e07-9d44-067716da3d6e","Type":"ContainerDied","Data":"2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851"}
Jul 12 18:53:59.435515 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.435451    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z" event={"ID":"348fbf1d-2f31-4e07-9d44-067716da3d6e","Type":"ContainerDied","Data":"32b3c584b61494874a7c39640595cff8317b8f788720b73a36d6744f794093a7"}
Jul 12 18:53:59.435515 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.435495    2562 scope.go:117] "RemoveContainer" containerID="2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851"
Jul 12 18:53:59.443949 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.443935    2562 scope.go:117] "RemoveContainer" containerID="2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851"
Jul 12 18:53:59.444188 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:53:59.444170    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851\": container with ID starting with 2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851 not found: ID does not exist" containerID="2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851"
Jul 12 18:53:59.444236 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.444195    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851"} err="failed to get container status \"2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851\": rpc error: code = NotFound desc = could not find container \"2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851\": container with ID starting with 2d75e114db26b77d1f44515877eac26ee811010799942d13be0004b44d454851 not found: ID does not exist"
Jul 12 18:53:59.456383 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.456358    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"]
Jul 12 18:53:59.460597 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.460575    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-443e5c6f-74c8ffc695-fnd2z"]
Jul 12 18:53:59.516965 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.516929    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/348fbf1d-2f31-4e07-9d44-067716da3d6e-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:53:59.516965 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.516961    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98bd5\" (UniqueName: \"kubernetes.io/projected/348fbf1d-2f31-4e07-9d44-067716da3d6e-kube-api-access-98bd5\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:53:59.517137 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.516976    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/348fbf1d-2f31-4e07-9d44-067716da3d6e-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:53:59.602181 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:53:59.602151    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="348fbf1d-2f31-4e07-9d44-067716da3d6e" path="/var/lib/kubelet/pods/348fbf1d-2f31-4e07-9d44-067716da3d6e/volumes"
Jul 12 18:54:04.679815 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:04.679775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:54:09.663584 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:09.663550    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"]
Jul 12 18:54:09.759614 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:09.759574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:54:24.683935 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:24.683896    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4" podUID="0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" containerName="main" containerID="cri-o://d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573" gracePeriod=45
Jul 12 18:54:24.928000 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:24.927975    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:54:25.041748 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.041710    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tmp-dir\") pod \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") "
Jul 12 18:54:25.041943 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.041772    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lr7w\" (UniqueName: \"kubernetes.io/projected/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-kube-api-access-7lr7w\") pod \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") "
Jul 12 18:54:25.041943 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.041800    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-home\") pod \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") "
Jul 12 18:54:25.041943 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.041826    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-dshm\") pod \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") "
Jul 12 18:54:25.041943 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.041864    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tls-certs\") pod \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") "
Jul 12 18:54:25.041943 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.041909    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-model-cache\") pod \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\" (UID: \"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6\") "
Jul 12 18:54:25.042202 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.042037    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" (UID: "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:54:25.042202 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.042132    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tmp-dir\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:54:25.042202 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.042146    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-home" (OuterVolumeSpecName: "home") pod "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" (UID: "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:54:25.042202 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.042195    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-model-cache" (OuterVolumeSpecName: "model-cache") pod "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" (UID: "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:54:25.044121 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.044096    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-dshm" (OuterVolumeSpecName: "dshm") pod "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" (UID: "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:54:25.044121 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.044112    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-kube-api-access-7lr7w" (OuterVolumeSpecName: "kube-api-access-7lr7w") pod "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" (UID: "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6"). InnerVolumeSpecName "kube-api-access-7lr7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:54:25.044121 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.044096    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" (UID: "0d9e9766-c443-41b1-b4a8-e8f9ed9008b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:54:25.142733 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.142702    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lr7w\" (UniqueName: \"kubernetes.io/projected/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-kube-api-access-7lr7w\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:54:25.142733 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.142730    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-home\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:54:25.142733 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.142740    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-dshm\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:54:25.142979 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.142750    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-tls-certs\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:54:25.142979 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.142759    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6-model-cache\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:54:25.541216 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.541187    2562 generic.go:356] "Generic (PLEG): container finished" podID="0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" containerID="d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573" exitCode=0
Jul 12 18:54:25.541393 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.541228    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"
Jul 12 18:54:25.541393 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.541277    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4" event={"ID":"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6","Type":"ContainerDied","Data":"d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573"}
Jul 12 18:54:25.541393 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.541319    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4" event={"ID":"0d9e9766-c443-41b1-b4a8-e8f9ed9008b6","Type":"ContainerDied","Data":"81f55d936f031c9824191940e1a04a3553d5bef20aa17f7947bad3888e83d3ff"}
Jul 12 18:54:25.541393 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.541336    2562 scope.go:117] "RemoveContainer" containerID="d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573"
Jul 12 18:54:25.549380 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.549362    2562 scope.go:117] "RemoveContainer" containerID="d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573"
Jul 12 18:54:25.549644 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:54:25.549624    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573\": container with ID starting with d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573 not found: ID does not exist" containerID="d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573"
Jul 12 18:54:25.549717 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.549651    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573"} err="failed to get container status \"d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573\": rpc error: code = NotFound desc = could not find container \"d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573\": container with ID starting with d9835b66c475c8443a9b0d57bbdd3d8716efcd42ba4f9f064c1f1900acb6e573 not found: ID does not exist"
Jul 12 18:54:25.560926 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.560902    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"]
Jul 12 18:54:25.564208 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.564188    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-443e5c6f/e2e-lifecycle-model-443e5c6f-kserve-6d8df7996c-49gm4"]
Jul 12 18:54:25.601635 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:25.601612    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" path="/var/lib/kubelet/pods/0d9e9766-c443-41b1-b4a8-e8f9ed9008b6/volumes"
Jul 12 18:54:46.192922 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:46.192892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:54:46.870206 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:46.870176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:54:52.299190 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:52.299156    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:54:57.983583 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:57.983543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:54:58.041354 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:58.041328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:54:58.327675 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:58.327589    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:54:59.534906 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:59.534873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:54:59.629290 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:59.629263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:54:59.729209 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:54:59.729172    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:00.443515 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:00.443481    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:00.542662 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:00.542632    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:00.642533 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:00.642488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:01.967398 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:01.967366    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:07.792568 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:07.792536    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:08.989867 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:08.989794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:14.161178 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:14.161142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:14.269770 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:14.269717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:14.380888 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:14.380857    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:17.607763 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:17.607737    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:55:17.616021 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:17.616001    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 18:55:24.914343 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:24.914309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:25.245069 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:25.245037    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:46.177601 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:46.177570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:49.947732 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:49.947693    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:52.381889 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:52.381855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:52.438163 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:52.438127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:52.723689 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:52.723663    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:53.005709 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:53.005629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:53.550598 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:53.550566    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:56.287684 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.287654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:56.355236 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.355202    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"]
Jul 12 18:55:56.355597 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.355580    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="348fbf1d-2f31-4e07-9d44-067716da3d6e" containerName="maas-api"
Jul 12 18:55:56.355648 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.355599    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="348fbf1d-2f31-4e07-9d44-067716da3d6e" containerName="maas-api"
Jul 12 18:55:56.355648 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.355617    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" containerName="main"
Jul 12 18:55:56.355648 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.355624    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" containerName="main"
Jul 12 18:55:56.355744 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.355675    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="348fbf1d-2f31-4e07-9d44-067716da3d6e" containerName="maas-api"
Jul 12 18:55:56.355744 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.355685    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d9e9766-c443-41b1-b4a8-e8f9ed9008b6" containerName="main"
Jul 12 18:55:56.358469 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.358442    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:56.360790 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.360769    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-58790d-serving-cert\""
Jul 12 18:55:56.369935 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.369916    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"]
Jul 12 18:55:56.378973 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.378950    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:56.396620 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.396591    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b9f88a-11f5-46fd-8030-6f545a74e673-openshift-service-ca\") pod \"maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:56.396737 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.396659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b9f88a-11f5-46fd-8030-6f545a74e673-maas-api-tls\") pod \"maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:56.396737 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.396717    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czhrz\" (UniqueName: \"kubernetes.io/projected/68b9f88a-11f5-46fd-8030-6f545a74e673-kube-api-access-czhrz\") pod \"maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:56.448201 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.448176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:56.497691 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.497656    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b9f88a-11f5-46fd-8030-6f545a74e673-maas-api-tls\") pod \"maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:56.497832 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.497740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czhrz\" (UniqueName: \"kubernetes.io/projected/68b9f88a-11f5-46fd-8030-6f545a74e673-kube-api-access-czhrz\") pod \"maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:56.497832 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.497790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b9f88a-11f5-46fd-8030-6f545a74e673-openshift-service-ca\") pod \"maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:56.497832 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:55:56.497802    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-serving-cert: secret "maas-api-e2e-inf-a-58790d-serving-cert" not found
Jul 12 18:55:56.497959 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:55:56.497874    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68b9f88a-11f5-46fd-8030-6f545a74e673-maas-api-tls podName:68b9f88a-11f5-46fd-8030-6f545a74e673 nodeName:}" failed. No retries permitted until 2026-07-12 18:55:56.99785692 +0000 UTC m=+1839.881207172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/68b9f88a-11f5-46fd-8030-6f545a74e673-maas-api-tls") pod "maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8" (UID: "68b9f88a-11f5-46fd-8030-6f545a74e673") : secret "maas-api-e2e-inf-a-58790d-serving-cert" not found
Jul 12 18:55:56.498553 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.498535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b9f88a-11f5-46fd-8030-6f545a74e673-openshift-service-ca\") pod \"maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:56.516780 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.516750    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czhrz\" (UniqueName: \"kubernetes.io/projected/68b9f88a-11f5-46fd-8030-6f545a74e673-kube-api-access-czhrz\") pod \"maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:56.710920 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.710834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:56.991206 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:56.991177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:57.001265 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:57.001238    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b9f88a-11f5-46fd-8030-6f545a74e673-maas-api-tls\") pod \"maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:57.003641 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:57.003614    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b9f88a-11f5-46fd-8030-6f545a74e673-maas-api-tls\") pod \"maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:57.091263 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:57.091233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:55:57.268948 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:57.268848    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:57.395431 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:57.395410    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"]
Jul 12 18:55:57.397856 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:55:57.397827    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68b9f88a_11f5_46fd_8030_6f545a74e673.slice/crio-ac27017f4b58d7212bcab1599a1f55caca2d2e0a6ec012c901685e8c6a5dabf5 WatchSource:0}: Error finding container ac27017f4b58d7212bcab1599a1f55caca2d2e0a6ec012c901685e8c6a5dabf5: Status 404 returned error can't find the container with id ac27017f4b58d7212bcab1599a1f55caca2d2e0a6ec012c901685e8c6a5dabf5
Jul 12 18:55:57.399109 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:57.399089    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 18:55:57.904191 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:57.904157    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8" event={"ID":"68b9f88a-11f5-46fd-8030-6f545a74e673","Type":"ContainerStarted","Data":"a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916"}
Jul 12 18:55:57.904337 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:57.904208    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8" event={"ID":"68b9f88a-11f5-46fd-8030-6f545a74e673","Type":"ContainerStarted","Data":"ac27017f4b58d7212bcab1599a1f55caca2d2e0a6ec012c901685e8c6a5dabf5"}
Jul 12 18:55:57.904337 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:57.904238    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:55:57.921158 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:55:57.921113    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8" podStartSLOduration=1.503949085 podStartE2EDuration="1.921097407s" totalImagesPullingTime="417.148322ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:55:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:55:57.920006512 +0000 UTC m=+1840.803356786" watchObservedRunningTime="2026-07-12 18:55:57.921097407 +0000 UTC m=+1840.804447681"
Jul 12 18:56:00.677658 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:00.677623    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:00.994013 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:00.993984    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:01.298045 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:01.297960    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:01.424359 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:01.424328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:03.914326 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:03.914297    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:56:04.213711 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.213624    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:04.312227 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.312198    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"]
Jul 12 18:56:04.313515 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.313483    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:04.315930 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.315907    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:04.317959 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.317938    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-47ad52-serving-cert\""
Jul 12 18:56:04.325058 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.324819    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"]
Jul 12 18:56:04.367744 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.367706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztjtc\" (UniqueName: \"kubernetes.io/projected/6374e261-9595-4cc8-b17d-907458b6e98b-kube-api-access-ztjtc\") pod \"maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:04.367915 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.367761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6374e261-9595-4cc8-b17d-907458b6e98b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:04.367915 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.367865    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6374e261-9595-4cc8-b17d-907458b6e98b-maas-api-tls\") pod \"maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:04.384099 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.384075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:04.469287 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.469198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztjtc\" (UniqueName: \"kubernetes.io/projected/6374e261-9595-4cc8-b17d-907458b6e98b-kube-api-access-ztjtc\") pod \"maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:04.469287 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.469253    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6374e261-9595-4cc8-b17d-907458b6e98b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:04.469453 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.469317    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6374e261-9595-4cc8-b17d-907458b6e98b-maas-api-tls\") pod \"maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:04.469453 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:56:04.469411    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-serving-cert: secret "maas-api-e2e-inf-b-47ad52-serving-cert" not found
Jul 12 18:56:04.469536 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:56:04.469482    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6374e261-9595-4cc8-b17d-907458b6e98b-maas-api-tls podName:6374e261-9595-4cc8-b17d-907458b6e98b nodeName:}" failed. No retries permitted until 2026-07-12 18:56:04.969448025 +0000 UTC m=+1847.852798280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6374e261-9595-4cc8-b17d-907458b6e98b-maas-api-tls") pod "maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85" (UID: "6374e261-9595-4cc8-b17d-907458b6e98b") : secret "maas-api-e2e-inf-b-47ad52-serving-cert" not found
Jul 12 18:56:04.469938 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.469919    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6374e261-9595-4cc8-b17d-907458b6e98b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:04.486111 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.486092    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztjtc\" (UniqueName: \"kubernetes.io/projected/6374e261-9595-4cc8-b17d-907458b6e98b-kube-api-access-ztjtc\") pod \"maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:04.739880 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.739843    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:04.852568 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.852532    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:04.955339 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.955310    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:04.973449 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.973424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6374e261-9595-4cc8-b17d-907458b6e98b-maas-api-tls\") pod \"maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:04.975783 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:04.975761    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6374e261-9595-4cc8-b17d-907458b6e98b-maas-api-tls\") pod \"maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:05.228954 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:05.228924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:05.357525 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:05.357281    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"]
Jul 12 18:56:05.359895 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:56:05.359868    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6374e261_9595_4cc8_b17d_907458b6e98b.slice/crio-0bb7c74b41313d2dad09cb70ff347a27173d79da66d631667f6b1a77b079ce19 WatchSource:0}: Error finding container 0bb7c74b41313d2dad09cb70ff347a27173d79da66d631667f6b1a77b079ce19: Status 404 returned error can't find the container with id 0bb7c74b41313d2dad09cb70ff347a27173d79da66d631667f6b1a77b079ce19
Jul 12 18:56:05.939101 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:05.939065    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85" event={"ID":"6374e261-9595-4cc8-b17d-907458b6e98b","Type":"ContainerStarted","Data":"9ea955261a81b739893bf8493a67311c007172fe617cfe98087daae0438f89b1"}
Jul 12 18:56:05.939297 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:05.939127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85" event={"ID":"6374e261-9595-4cc8-b17d-907458b6e98b","Type":"ContainerStarted","Data":"0bb7c74b41313d2dad09cb70ff347a27173d79da66d631667f6b1a77b079ce19"}
Jul 12 18:56:05.939297 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:05.939152    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:05.956617 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:05.956561    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85" podStartSLOduration=1.587934825 podStartE2EDuration="1.956185574s" totalImagesPullingTime="368.250749ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:56:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:56:05.955058899 +0000 UTC m=+1848.838409173" watchObservedRunningTime="2026-07-12 18:56:05.956185574 +0000 UTC m=+1848.839535847"
Jul 12 18:56:08.698349 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:08.698307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:11.954631 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:11.954600    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:56:17.680429 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:17.680399    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:17.781359 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:17.781328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:18.919479 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.919030    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"]
Jul 12 18:56:18.926146 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.926113    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:18.928731 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.928707    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-58790d\"/\"openshift-service-ca.crt\""
Jul 12 18:56:18.929354 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.929331    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-58790d\"/\"test-model-58790d-kserve-self-signed-certs\""
Jul 12 18:56:18.929354 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.929346    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-58790d\"/\"default-dockercfg-mmk4w\""
Jul 12 18:56:18.929602 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.929349    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-58790d\"/\"kube-root-ca.crt\""
Jul 12 18:56:18.929690 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.929637    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"]
Jul 12 18:56:18.943307 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.943285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:18.997335 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.997299    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsdck\" (UniqueName: \"kubernetes.io/projected/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-kube-api-access-hsdck\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:18.997497 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.997347    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-home\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:18.997497 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.997369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-dshm\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:18.997497 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.997400    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tls-certs\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:18.997497 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.997438    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tmp-dir\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:18.997632 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:18.997519    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-model-cache\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.098286 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.098246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tmp-dir\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.098513 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.098310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-model-cache\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.098513 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.098349    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsdck\" (UniqueName: \"kubernetes.io/projected/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-kube-api-access-hsdck\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.098513 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.098384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-home\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.098720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.098519    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-dshm\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.098720 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.098559    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tls-certs\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.098829 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.098713    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tmp-dir\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.098829 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.098752    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-model-cache\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.098829 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.098806    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-home\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.100879 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.100854    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-dshm\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.101031 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.101017    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tls-certs\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.105954 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.105932    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsdck\" (UniqueName: \"kubernetes.io/projected/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-kube-api-access-hsdck\") pod \"test-model-58790d-kserve-696cd767bc-mftqs\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") " pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.236372 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.236334    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:19.362985 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:19.362958    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"]
Jul 12 18:56:19.364417 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:56:19.364393    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd12d22e9_d1af_4cd9_8557_b7cfe5f92c75.slice/crio-1ff437f558f4a2fd7f8327e93d91150f71dd30006dd16c121573897b7e3feadf WatchSource:0}: Error finding container 1ff437f558f4a2fd7f8327e93d91150f71dd30006dd16c121573897b7e3feadf: Status 404 returned error can't find the container with id 1ff437f558f4a2fd7f8327e93d91150f71dd30006dd16c121573897b7e3feadf
Jul 12 18:56:20.002771 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:20.002738    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs" event={"ID":"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75","Type":"ContainerStarted","Data":"482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2"}
Jul 12 18:56:20.002771 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:20.002776    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs" event={"ID":"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75","Type":"ContainerStarted","Data":"1ff437f558f4a2fd7f8327e93d91150f71dd30006dd16c121573897b7e3feadf"}
Jul 12 18:56:20.017545 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:20.017497    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs" podStartSLOduration=2.017483452 podStartE2EDuration="2.017483452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:56:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:56:20.015985213 +0000 UTC m=+1862.899335490" watchObservedRunningTime="2026-07-12 18:56:20.017483452 +0000 UTC m=+1862.900833723"
Jul 12 18:56:25.899304 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:25.899269    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:26.000157 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:26.000122    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:56:29.236602 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:29.236538    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:29.237067 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:29.236624    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:29.248904 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:29.248885    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:30.052867 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:30.052836    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:56:50.241827 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:56:50.241745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:12.219383 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:12.219349    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"]
Jul 12 18:57:12.238274 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:12.238244    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:13.449698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:13.449665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:13.581947 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:13.581914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:13.970039 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:13.970006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:14.196504 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.196412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:14.296991 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.296956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:14.423933 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.423893    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"]
Jul 12 18:57:14.424146 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.424120    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8" podUID="68b9f88a-11f5-46fd-8030-6f545a74e673" containerName="maas-api" containerID="cri-o://a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916" gracePeriod=30
Jul 12 18:57:14.445809 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.445775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:14.668663 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.668634    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:57:14.795061 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.795035    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czhrz\" (UniqueName: \"kubernetes.io/projected/68b9f88a-11f5-46fd-8030-6f545a74e673-kube-api-access-czhrz\") pod \"68b9f88a-11f5-46fd-8030-6f545a74e673\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") "
Jul 12 18:57:14.795061 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.795068    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b9f88a-11f5-46fd-8030-6f545a74e673-openshift-service-ca\") pod \"68b9f88a-11f5-46fd-8030-6f545a74e673\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") "
Jul 12 18:57:14.795317 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.795116    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b9f88a-11f5-46fd-8030-6f545a74e673-maas-api-tls\") pod \"68b9f88a-11f5-46fd-8030-6f545a74e673\" (UID: \"68b9f88a-11f5-46fd-8030-6f545a74e673\") "
Jul 12 18:57:14.795487 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.795440    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b9f88a-11f5-46fd-8030-6f545a74e673-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68b9f88a-11f5-46fd-8030-6f545a74e673" (UID: "68b9f88a-11f5-46fd-8030-6f545a74e673"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:57:14.797177 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.797151    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68b9f88a-11f5-46fd-8030-6f545a74e673-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68b9f88a-11f5-46fd-8030-6f545a74e673" (UID: "68b9f88a-11f5-46fd-8030-6f545a74e673"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:57:14.797272 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.797253    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68b9f88a-11f5-46fd-8030-6f545a74e673-kube-api-access-czhrz" (OuterVolumeSpecName: "kube-api-access-czhrz") pod "68b9f88a-11f5-46fd-8030-6f545a74e673" (UID: "68b9f88a-11f5-46fd-8030-6f545a74e673"). InnerVolumeSpecName "kube-api-access-czhrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:57:14.896185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.896099    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b9f88a-11f5-46fd-8030-6f545a74e673-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:14.896185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.896147    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czhrz\" (UniqueName: \"kubernetes.io/projected/68b9f88a-11f5-46fd-8030-6f545a74e673-kube-api-access-czhrz\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:14.896185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:14.896158    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b9f88a-11f5-46fd-8030-6f545a74e673-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:15.231636 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:15.231600    2562 generic.go:356] "Generic (PLEG): container finished" podID="68b9f88a-11f5-46fd-8030-6f545a74e673" containerID="a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916" exitCode=0
Jul 12 18:57:15.231813 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:15.231647    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"
Jul 12 18:57:15.231813 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:15.231675    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8" event={"ID":"68b9f88a-11f5-46fd-8030-6f545a74e673","Type":"ContainerDied","Data":"a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916"}
Jul 12 18:57:15.231813 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:15.231727    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8" event={"ID":"68b9f88a-11f5-46fd-8030-6f545a74e673","Type":"ContainerDied","Data":"ac27017f4b58d7212bcab1599a1f55caca2d2e0a6ec012c901685e8c6a5dabf5"}
Jul 12 18:57:15.231813 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:15.231755    2562 scope.go:117] "RemoveContainer" containerID="a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916"
Jul 12 18:57:15.240471 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:15.240436    2562 scope.go:117] "RemoveContainer" containerID="a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916"
Jul 12 18:57:15.240730 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:57:15.240712    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916\": container with ID starting with a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916 not found: ID does not exist" containerID="a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916"
Jul 12 18:57:15.240781 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:15.240738    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916"} err="failed to get container status \"a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916\": rpc error: code = NotFound desc = could not find container \"a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916\": container with ID starting with a8a4703f86b551d1e2442623929fc48dd1e512f4e760661f36ddebde4ebb7916 not found: ID does not exist"
Jul 12 18:57:15.251963 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:15.251937    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"]
Jul 12 18:57:15.255736 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:15.255711    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58790d-5c849fc86b-kd5c8"]
Jul 12 18:57:15.602356 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:15.602281    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68b9f88a-11f5-46fd-8030-6f545a74e673" path="/var/lib/kubelet/pods/68b9f88a-11f5-46fd-8030-6f545a74e673/volumes"
Jul 12 18:57:19.047085 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:19.047053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:27.239548 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.239487    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs" podUID="d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" containerName="main" containerID="cri-o://482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2" gracePeriod=45
Jul 12 18:57:27.486891 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.486864    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:57:27.592406 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592332    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsdck\" (UniqueName: \"kubernetes.io/projected/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-kube-api-access-hsdck\") pod \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") "
Jul 12 18:57:27.592406 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592371    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tmp-dir\") pod \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") "
Jul 12 18:57:27.592406 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592389    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-home\") pod \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") "
Jul 12 18:57:27.592676 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592434    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tls-certs\") pod \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") "
Jul 12 18:57:27.592676 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592487    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-dshm\") pod \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") "
Jul 12 18:57:27.592676 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592533    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-model-cache\") pod \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\" (UID: \"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75\") "
Jul 12 18:57:27.592676 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592665    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" (UID: "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:57:27.592880 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592711    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-home" (OuterVolumeSpecName: "home") pod "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" (UID: "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:57:27.592880 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592802    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tmp-dir\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:27.592880 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592845    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-home\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:27.592880 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.592852    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-model-cache" (OuterVolumeSpecName: "model-cache") pod "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" (UID: "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:57:27.594610 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.594581    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-kube-api-access-hsdck" (OuterVolumeSpecName: "kube-api-access-hsdck") pod "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" (UID: "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75"). InnerVolumeSpecName "kube-api-access-hsdck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:57:27.594610 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.594582    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-dshm" (OuterVolumeSpecName: "dshm") pod "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" (UID: "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:57:27.594758 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.594618    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" (UID: "d12d22e9-d1af-4cd9-8557-b7cfe5f92c75"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:57:27.694116 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.694085    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-dshm\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:27.694116 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.694114    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-model-cache\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:27.694298 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.694129    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsdck\" (UniqueName: \"kubernetes.io/projected/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-kube-api-access-hsdck\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:27.694298 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:27.694140    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75-tls-certs\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:28.287141 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:28.287108    2562 generic.go:356] "Generic (PLEG): container finished" podID="d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" containerID="482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2" exitCode=0
Jul 12 18:57:28.287141 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:28.287145    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"
Jul 12 18:57:28.287641 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:28.287184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs" event={"ID":"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75","Type":"ContainerDied","Data":"482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2"}
Jul 12 18:57:28.287641 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:28.287225    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs" event={"ID":"d12d22e9-d1af-4cd9-8557-b7cfe5f92c75","Type":"ContainerDied","Data":"1ff437f558f4a2fd7f8327e93d91150f71dd30006dd16c121573897b7e3feadf"}
Jul 12 18:57:28.287641 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:28.287242    2562 scope.go:117] "RemoveContainer" containerID="482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2"
Jul 12 18:57:28.295440 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:28.295423    2562 scope.go:117] "RemoveContainer" containerID="482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2"
Jul 12 18:57:28.295718 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:57:28.295701    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2\": container with ID starting with 482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2 not found: ID does not exist" containerID="482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2"
Jul 12 18:57:28.295785 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:28.295725    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2"} err="failed to get container status \"482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2\": rpc error: code = NotFound desc = could not find container \"482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2\": container with ID starting with 482a382276574ee734fa432fdb90fffae070df3b415724580ed09eaaf7dedfb2 not found: ID does not exist"
Jul 12 18:57:28.302533 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:28.302503    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"]
Jul 12 18:57:28.304571 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:28.304549    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-58790d/test-model-58790d-kserve-696cd767bc-mftqs"]
Jul 12 18:57:29.601621 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:29.601586    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" path="/var/lib/kubelet/pods/d12d22e9-d1af-4cd9-8557-b7cfe5f92c75/volumes"
Jul 12 18:57:55.714393 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:55.714356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:55.978023 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:55.977991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:56.367054 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:56.366970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:56.791592 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:56.791556    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:57.042448 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:57.042361    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:57.138899 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:57.138865    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:57.266763 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:57.266725    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"]
Jul 12 18:57:57.267032 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:57.267007    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85" podUID="6374e261-9595-4cc8-b17d-907458b6e98b" containerName="maas-api" containerID="cri-o://9ea955261a81b739893bf8493a67311c007172fe617cfe98087daae0438f89b1" gracePeriod=30
Jul 12 18:57:57.302554 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:57.302446    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:57:58.409948 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.409913    2562 generic.go:356] "Generic (PLEG): container finished" podID="6374e261-9595-4cc8-b17d-907458b6e98b" containerID="9ea955261a81b739893bf8493a67311c007172fe617cfe98087daae0438f89b1" exitCode=0
Jul 12 18:57:58.410410 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.409963    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85" event={"ID":"6374e261-9595-4cc8-b17d-907458b6e98b","Type":"ContainerDied","Data":"9ea955261a81b739893bf8493a67311c007172fe617cfe98087daae0438f89b1"}
Jul 12 18:57:58.509114 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.509087    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:57:58.658858 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.658767    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6374e261-9595-4cc8-b17d-907458b6e98b-maas-api-tls\") pod \"6374e261-9595-4cc8-b17d-907458b6e98b\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") "
Jul 12 18:57:58.659010 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.658857    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6374e261-9595-4cc8-b17d-907458b6e98b-openshift-service-ca\") pod \"6374e261-9595-4cc8-b17d-907458b6e98b\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") "
Jul 12 18:57:58.659010 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.658900    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztjtc\" (UniqueName: \"kubernetes.io/projected/6374e261-9595-4cc8-b17d-907458b6e98b-kube-api-access-ztjtc\") pod \"6374e261-9595-4cc8-b17d-907458b6e98b\" (UID: \"6374e261-9595-4cc8-b17d-907458b6e98b\") "
Jul 12 18:57:58.659204 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.659184    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6374e261-9595-4cc8-b17d-907458b6e98b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6374e261-9595-4cc8-b17d-907458b6e98b" (UID: "6374e261-9595-4cc8-b17d-907458b6e98b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:57:58.660956 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.660932    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6374e261-9595-4cc8-b17d-907458b6e98b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6374e261-9595-4cc8-b17d-907458b6e98b" (UID: "6374e261-9595-4cc8-b17d-907458b6e98b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:57:58.661066 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.661045    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6374e261-9595-4cc8-b17d-907458b6e98b-kube-api-access-ztjtc" (OuterVolumeSpecName: "kube-api-access-ztjtc") pod "6374e261-9595-4cc8-b17d-907458b6e98b" (UID: "6374e261-9595-4cc8-b17d-907458b6e98b"). InnerVolumeSpecName "kube-api-access-ztjtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:57:58.760323 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.760289    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztjtc\" (UniqueName: \"kubernetes.io/projected/6374e261-9595-4cc8-b17d-907458b6e98b-kube-api-access-ztjtc\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:58.760323 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.760317    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6374e261-9595-4cc8-b17d-907458b6e98b-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:58.760323 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:58.760326    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6374e261-9595-4cc8-b17d-907458b6e98b-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:57:59.415188 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:59.415155    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85" event={"ID":"6374e261-9595-4cc8-b17d-907458b6e98b","Type":"ContainerDied","Data":"0bb7c74b41313d2dad09cb70ff347a27173d79da66d631667f6b1a77b079ce19"}
Jul 12 18:57:59.415188 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:59.415192    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"
Jul 12 18:57:59.415709 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:59.415225    2562 scope.go:117] "RemoveContainer" containerID="9ea955261a81b739893bf8493a67311c007172fe617cfe98087daae0438f89b1"
Jul 12 18:57:59.442005 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:59.441977    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"]
Jul 12 18:57:59.445266 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:59.445246    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47ad52-6fc7b4fb65-5fr85"]
Jul 12 18:57:59.602619 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:57:59.602588    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6374e261-9595-4cc8-b17d-907458b6e98b" path="/var/lib/kubelet/pods/6374e261-9595-4cc8-b17d-907458b6e98b/volumes"
Jul 12 18:58:01.336502 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:01.336447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:01.443105 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:01.443070    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:02.816737 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:02.816701    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:02.920675 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:02.920630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:03.143959 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:03.143860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:03.703158 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:03.703126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:04.032333 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:04.032297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:04.250378 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:04.250329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:06.960139 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:06.960100    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:07.057072 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.057034    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:07.326900 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.326867    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"]
Jul 12 18:58:07.327175 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.327162    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68b9f88a-11f5-46fd-8030-6f545a74e673" containerName="maas-api"
Jul 12 18:58:07.327240 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.327176    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="68b9f88a-11f5-46fd-8030-6f545a74e673" containerName="maas-api"
Jul 12 18:58:07.327240 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.327192    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6374e261-9595-4cc8-b17d-907458b6e98b" containerName="maas-api"
Jul 12 18:58:07.327240 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.327197    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6374e261-9595-4cc8-b17d-907458b6e98b" containerName="maas-api"
Jul 12 18:58:07.327240 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.327209    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" containerName="main"
Jul 12 18:58:07.327240 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.327215    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" containerName="main"
Jul 12 18:58:07.327521 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.327262    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="68b9f88a-11f5-46fd-8030-6f545a74e673" containerName="maas-api"
Jul 12 18:58:07.327521 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.327269    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d12d22e9-d1af-4cd9-8557-b7cfe5f92c75" containerName="main"
Jul 12 18:58:07.327521 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.327275    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6374e261-9595-4cc8-b17d-907458b6e98b" containerName="maas-api"
Jul 12 18:58:07.331386 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.331363    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.333837 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.333817    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6ee31b-serving-cert\""
Jul 12 18:58:07.339185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.339093    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"]
Jul 12 18:58:07.339401 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.339303    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-maas-api-tls\") pod \"maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.339581 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.339552    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-openshift-service-ca\") pod \"maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.339655 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.339619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s4f2\" (UniqueName: \"kubernetes.io/projected/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-kube-api-access-5s4f2\") pod \"maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.354962 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.354924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:07.440139 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.440110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s4f2\" (UniqueName: \"kubernetes.io/projected/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-kube-api-access-5s4f2\") pod \"maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.440343 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.440188    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-maas-api-tls\") pod \"maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.440343 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.440209    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-openshift-service-ca\") pod \"maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.440492 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:58:07.440343    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-serving-cert: secret "maas-api-e2e-api-a-6ee31b-serving-cert" not found
Jul 12 18:58:07.440492 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:58:07.440417    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-maas-api-tls podName:45bf3f7a-bc96-40e2-9cc2-3d3faa393f12 nodeName:}" failed. No retries permitted until 2026-07-12 18:58:07.94039539 +0000 UTC m=+1970.823745659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-maas-api-tls") pod "maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76" (UID: "45bf3f7a-bc96-40e2-9cc2-3d3faa393f12") : secret "maas-api-e2e-api-a-6ee31b-serving-cert" not found
Jul 12 18:58:07.440866 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.440846    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-openshift-service-ca\") pod \"maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.448840 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.448818    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s4f2\" (UniqueName: \"kubernetes.io/projected/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-kube-api-access-5s4f2\") pod \"maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.762334 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.762291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:07.864165 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.864129    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:07.943885 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.943850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-maas-api-tls\") pod \"maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.946243 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.946201    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-maas-api-tls\") pod \"maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:07.964844 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:07.964806    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:08.243269 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:08.243229    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:08.382648 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:08.382611    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"]
Jul 12 18:58:08.384933 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:58:08.384900    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45bf3f7a_bc96_40e2_9cc2_3d3faa393f12.slice/crio-0485d09b01330716d483e594b4b2db0156a5df4d7bd7559ab90ed84728ec69f3 WatchSource:0}: Error finding container 0485d09b01330716d483e594b4b2db0156a5df4d7bd7559ab90ed84728ec69f3: Status 404 returned error can't find the container with id 0485d09b01330716d483e594b4b2db0156a5df4d7bd7559ab90ed84728ec69f3
Jul 12 18:58:08.452832 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:08.452800    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76" event={"ID":"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12","Type":"ContainerStarted","Data":"0485d09b01330716d483e594b4b2db0156a5df4d7bd7559ab90ed84728ec69f3"}
Jul 12 18:58:09.459308 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:09.459271    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76" event={"ID":"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12","Type":"ContainerStarted","Data":"156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107"}
Jul 12 18:58:09.459732 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:09.459358    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:09.480935 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:09.480890    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76" podStartSLOduration=2.101831412 podStartE2EDuration="2.480874548s" totalImagesPullingTime="379.043136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:58:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:58:09.47933209 +0000 UTC m=+1972.362682363" watchObservedRunningTime="2026-07-12 18:58:09.480874548 +0000 UTC m=+1972.364224822"
Jul 12 18:58:11.448226 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:11.448196    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:11.915862 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:11.915831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:12.219152 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:12.219010    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:12.319470 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:12.319427    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:14.980558 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:14.980520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:15.089650 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.089606    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:15.107112 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.107084    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"]
Jul 12 18:58:15.110635 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.110611    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.112878 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.112854    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-498f3f-serving-cert\""
Jul 12 18:58:15.122333 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.122309    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"]
Jul 12 18:58:15.185025 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.184994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:15.204320 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.204283    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-maas-api-tls\") pod \"maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.204496 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.204327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-openshift-service-ca\") pod \"maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.204496 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.204437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7rkz\" (UniqueName: \"kubernetes.io/projected/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-kube-api-access-f7rkz\") pod \"maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.280707 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.280612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:15.305842 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.305817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-maas-api-tls\") pod \"maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.305984 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.305854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-openshift-service-ca\") pod \"maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.305984 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.305901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7rkz\" (UniqueName: \"kubernetes.io/projected/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-kube-api-access-f7rkz\") pod \"maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.306624 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.306598    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-openshift-service-ca\") pod \"maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.308351 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.308301    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-maas-api-tls\") pod \"maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.313835 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.313812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7rkz\" (UniqueName: \"kubernetes.io/projected/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-kube-api-access-f7rkz\") pod \"maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.421903 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.421863    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:15.471318 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.471293    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:15.551769 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.551746    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"]
Jul 12 18:58:15.553843 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:58:15.553811    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1daf47e4_4fb0_41b4_a81c_3d27a9ff375b.slice/crio-0bb449a1adf479f98e6af63c00628e4b3cbe0ca244678d402452cc391b887665 WatchSource:0}: Error finding container 0bb449a1adf479f98e6af63c00628e4b3cbe0ca244678d402452cc391b887665: Status 404 returned error can't find the container with id 0bb449a1adf479f98e6af63c00628e4b3cbe0ca244678d402452cc391b887665
Jul 12 18:58:15.633447 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.633413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:15.734874 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:15.734844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:16.490159 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:16.490122    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn" event={"ID":"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b","Type":"ContainerStarted","Data":"6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2"}
Jul 12 18:58:16.490567 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:16.490180    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn" event={"ID":"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b","Type":"ContainerStarted","Data":"0bb449a1adf479f98e6af63c00628e4b3cbe0ca244678d402452cc391b887665"}
Jul 12 18:58:16.490567 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:16.490228    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:16.506951 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:16.506909    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn" podStartSLOduration=1.216028428 podStartE2EDuration="1.506893788s" totalImagesPullingTime="290.86536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:58:16.505939809 +0000 UTC m=+1979.389290084" watchObservedRunningTime="2026-07-12 18:58:16.506893788 +0000 UTC m=+1979.390244064"
Jul 12 18:58:19.464911 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:19.464873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:22.502860 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:22.502829    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:27.471395 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:27.471355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:27.843917 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:27.843888    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:27.944869 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:27.944839    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:28.045164 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.045132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:28.255275 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.255244    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"]
Jul 12 18:58:28.255542 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.255517    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76" podUID="45bf3f7a-bc96-40e2-9cc2-3d3faa393f12" containerName="maas-api" containerID="cri-o://156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107" gracePeriod=30
Jul 12 18:58:28.284334 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.284296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:28.503117 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.503089    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:28.544805 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.544734    2562 generic.go:356] "Generic (PLEG): container finished" podID="45bf3f7a-bc96-40e2-9cc2-3d3faa393f12" containerID="156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107" exitCode=0
Jul 12 18:58:28.544805 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.544784    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"
Jul 12 18:58:28.544966 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.544821    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76" event={"ID":"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12","Type":"ContainerDied","Data":"156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107"}
Jul 12 18:58:28.544966 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.544867    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76" event={"ID":"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12","Type":"ContainerDied","Data":"0485d09b01330716d483e594b4b2db0156a5df4d7bd7559ab90ed84728ec69f3"}
Jul 12 18:58:28.544966 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.544893    2562 scope.go:117] "RemoveContainer" containerID="156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107"
Jul 12 18:58:28.552906 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.552878    2562 scope.go:117] "RemoveContainer" containerID="156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107"
Jul 12 18:58:28.553166 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:58:28.553147    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107\": container with ID starting with 156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107 not found: ID does not exist" containerID="156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107"
Jul 12 18:58:28.553228 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.553173    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107"} err="failed to get container status \"156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107\": rpc error: code = NotFound desc = could not find container \"156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107\": container with ID starting with 156dba3e1dcd6db3f496634af8faee9a044c816cf1ac69836435ff76aa861107 not found: ID does not exist"
Jul 12 18:58:28.622294 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.622275    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-openshift-service-ca\") pod \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") "
Jul 12 18:58:28.622380 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.622317    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-maas-api-tls\") pod \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") "
Jul 12 18:58:28.622380 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.622340    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5s4f2\" (UniqueName: \"kubernetes.io/projected/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-kube-api-access-5s4f2\") pod \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\" (UID: \"45bf3f7a-bc96-40e2-9cc2-3d3faa393f12\") "
Jul 12 18:58:28.622646 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.622624    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "45bf3f7a-bc96-40e2-9cc2-3d3faa393f12" (UID: "45bf3f7a-bc96-40e2-9cc2-3d3faa393f12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:58:28.624448 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.624425    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "45bf3f7a-bc96-40e2-9cc2-3d3faa393f12" (UID: "45bf3f7a-bc96-40e2-9cc2-3d3faa393f12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:58:28.624448 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.624436    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-kube-api-access-5s4f2" (OuterVolumeSpecName: "kube-api-access-5s4f2") pod "45bf3f7a-bc96-40e2-9cc2-3d3faa393f12" (UID: "45bf3f7a-bc96-40e2-9cc2-3d3faa393f12"). InnerVolumeSpecName "kube-api-access-5s4f2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:58:28.723251 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.723212    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:58:28.723251 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.723245    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:58:28.723251 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.723257    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s4f2\" (UniqueName: \"kubernetes.io/projected/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12-kube-api-access-5s4f2\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:58:28.865985 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.865958    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"]
Jul 12 18:58:28.869353 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:28.869328    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ee31b-76c6cf7bb8-xdr76"]
Jul 12 18:58:29.601844 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:29.601814    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45bf3f7a-bc96-40e2-9cc2-3d3faa393f12" path="/var/lib/kubelet/pods/45bf3f7a-bc96-40e2-9cc2-3d3faa393f12/volumes"
Jul 12 18:58:33.930784 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:33.930751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:33.994823 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:33.994788    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:34.272588 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:34.272553    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:34.560924 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:34.560843    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:34.940217 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:34.940133    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:35.039476 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.039423    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:35.142355 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.142327    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:35.243193 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.243160    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"]
Jul 12 18:58:35.243450 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.243418    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn" podUID="1daf47e4-4fb0-41b4-a81c-3d27a9ff375b" containerName="maas-api" containerID="cri-o://6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2" gracePeriod=30
Jul 12 18:58:35.275594 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.275566    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:35.482520 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.482493    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:35.574505 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.574399    2562 generic.go:356] "Generic (PLEG): container finished" podID="1daf47e4-4fb0-41b4-a81c-3d27a9ff375b" containerID="6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2" exitCode=0
Jul 12 18:58:35.574505 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.574446    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"
Jul 12 18:58:35.574675 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.574496    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn" event={"ID":"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b","Type":"ContainerDied","Data":"6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2"}
Jul 12 18:58:35.574675 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.574554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn" event={"ID":"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b","Type":"ContainerDied","Data":"0bb449a1adf479f98e6af63c00628e4b3cbe0ca244678d402452cc391b887665"}
Jul 12 18:58:35.574675 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.574577    2562 scope.go:117] "RemoveContainer" containerID="6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2"
Jul 12 18:58:35.579594 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.579577    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-maas-api-tls\") pod \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") "
Jul 12 18:58:35.579698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.579603    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-openshift-service-ca\") pod \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") "
Jul 12 18:58:35.579698 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.579684    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7rkz\" (UniqueName: \"kubernetes.io/projected/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-kube-api-access-f7rkz\") pod \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\" (UID: \"1daf47e4-4fb0-41b4-a81c-3d27a9ff375b\") "
Jul 12 18:58:35.580042 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.580018    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1daf47e4-4fb0-41b4-a81c-3d27a9ff375b" (UID: "1daf47e4-4fb0-41b4-a81c-3d27a9ff375b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 18:58:35.581594 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.581567    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1daf47e4-4fb0-41b4-a81c-3d27a9ff375b" (UID: "1daf47e4-4fb0-41b4-a81c-3d27a9ff375b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:58:35.581673 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.581623    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-kube-api-access-f7rkz" (OuterVolumeSpecName: "kube-api-access-f7rkz") pod "1daf47e4-4fb0-41b4-a81c-3d27a9ff375b" (UID: "1daf47e4-4fb0-41b4-a81c-3d27a9ff375b"). InnerVolumeSpecName "kube-api-access-f7rkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:58:35.582495 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.582477    2562 scope.go:117] "RemoveContainer" containerID="6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2"
Jul 12 18:58:35.582749 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:58:35.582725    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2\": container with ID starting with 6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2 not found: ID does not exist" containerID="6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2"
Jul 12 18:58:35.582805 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.582758    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2"} err="failed to get container status \"6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2\": rpc error: code = NotFound desc = could not find container \"6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2\": container with ID starting with 6242ad4c12c1bdb7dd98d760142faa53e935e6db6ce0a7985eeb5169453802c2 not found: ID does not exist"
Jul 12 18:58:35.680301 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.680262    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7rkz\" (UniqueName: \"kubernetes.io/projected/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-kube-api-access-f7rkz\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:58:35.680301 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.680295    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:58:35.680301 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.680309    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 18:58:35.889957 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.889873    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"]
Jul 12 18:58:35.893032 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:35.893010    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-498f3f-78fcbc88d7-rm2rn"]
Jul 12 18:58:37.602745 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:37.602712    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1daf47e4-4fb0-41b4-a81c-3d27a9ff375b" path="/var/lib/kubelet/pods/1daf47e4-4fb0-41b4-a81c-3d27a9ff375b/volumes"
Jul 12 18:58:41.263908 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:41.263876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:41.428866 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:41.428828    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:41.560000 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:41.559909    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:42.090185 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:42.090155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:42.385863 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:42.385787    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:42.525899 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:42.525863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:45.203101 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.203043    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:45.302975 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.302937    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:45.315066 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.315026    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"]
Jul 12 18:58:45.315332 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.315318    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1daf47e4-4fb0-41b4-a81c-3d27a9ff375b" containerName="maas-api"
Jul 12 18:58:45.315381 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.315334    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1daf47e4-4fb0-41b4-a81c-3d27a9ff375b" containerName="maas-api"
Jul 12 18:58:45.315381 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.315353    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45bf3f7a-bc96-40e2-9cc2-3d3faa393f12" containerName="maas-api"
Jul 12 18:58:45.315381 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.315358    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="45bf3f7a-bc96-40e2-9cc2-3d3faa393f12" containerName="maas-api"
Jul 12 18:58:45.315495 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.315431    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="45bf3f7a-bc96-40e2-9cc2-3d3faa393f12" containerName="maas-api"
Jul 12 18:58:45.315495 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.315443    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1daf47e4-4fb0-41b4-a81c-3d27a9ff375b" containerName="maas-api"
Jul 12 18:58:45.319678 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.319662    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:45.321718 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.321701    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-e0bc69-serving-cert\""
Jul 12 18:58:45.329019 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.328992    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"]
Jul 12 18:58:45.412846 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.412814    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:45.468155 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.468068    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qzj8\" (UniqueName: \"kubernetes.io/projected/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-kube-api-access-6qzj8\") pod \"maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:45.468155 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.468120    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-maas-api-tls\") pod \"maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:45.468347 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.468173    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:45.569143 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.569109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qzj8\" (UniqueName: \"kubernetes.io/projected/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-kube-api-access-6qzj8\") pod \"maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:45.569314 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.569157    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-maas-api-tls\") pod \"maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:45.569314 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.569191    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:45.569397 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:58:45.569319    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-serving-cert: secret "maas-api-e2e-shared-a-e0bc69-serving-cert" not found
Jul 12 18:58:45.569397 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:58:45.569385    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-maas-api-tls podName:4f5356ea-c2f9-4b07-8f4f-7508e0f69d44 nodeName:}" failed. No retries permitted until 2026-07-12 18:58:46.069366372 +0000 UTC m=+2008.952716641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-maas-api-tls") pod "maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf" (UID: "4f5356ea-c2f9-4b07-8f4f-7508e0f69d44") : secret "maas-api-e2e-shared-a-e0bc69-serving-cert" not found
Jul 12 18:58:45.569829 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.569812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:45.578089 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.578058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qzj8\" (UniqueName: \"kubernetes.io/projected/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-kube-api-access-6qzj8\") pod \"maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:45.869106 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.869074    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:45.969315 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:45.969285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:46.069591 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:46.069562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:46.073681 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:46.073652    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-maas-api-tls\") pod \"maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:46.076077 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:46.076058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-maas-api-tls\") pod \"maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:46.230509 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:46.230472    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:46.355114 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:46.355094    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"]
Jul 12 18:58:46.357569 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:58:46.357544    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5356ea_c2f9_4b07_8f4f_7508e0f69d44.slice/crio-e7a815716d14a64875f8d05b9029b004b1fcae8a362dae496fe1e5b93f24de2f WatchSource:0}: Error finding container e7a815716d14a64875f8d05b9029b004b1fcae8a362dae496fe1e5b93f24de2f: Status 404 returned error can't find the container with id e7a815716d14a64875f8d05b9029b004b1fcae8a362dae496fe1e5b93f24de2f
Jul 12 18:58:46.619542 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:46.619437    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf" event={"ID":"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44","Type":"ContainerStarted","Data":"e7a815716d14a64875f8d05b9029b004b1fcae8a362dae496fe1e5b93f24de2f"}
Jul 12 18:58:47.624365 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:47.624329    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf" event={"ID":"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44","Type":"ContainerStarted","Data":"5726f766f615f3045efb686deb421b39c1c4d07a9ce1073f4c2f3bbb937bc7c9"}
Jul 12 18:58:47.624791 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:47.624480    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:47.641135 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:47.641093    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf" podStartSLOduration=2.275116722 podStartE2EDuration="2.641081171s" totalImagesPullingTime="365.964449ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:58:47.640259341 +0000 UTC m=+2010.523609616" watchObservedRunningTime="2026-07-12 18:58:47.641081171 +0000 UTC m=+2010.524431447"
Jul 12 18:58:49.699322 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:49.699289    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:50.017919 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:50.017886    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:50.309067 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:50.308981    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:50.433354 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:50.433324    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:53.322738 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.322702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:53.422409 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.422375    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:53.620916 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.620845    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"]
Jul 12 18:58:53.624350 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.624327    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:53.626560 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.626539    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-92be65-serving-cert\""
Jul 12 18:58:53.634804 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.634772    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"]
Jul 12 18:58:53.637154 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.637132    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 18:58:53.662959 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.662927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:53.745222 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.745184    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:53.745416 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.745255    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djv4b\" (UniqueName: \"kubernetes.io/projected/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-kube-api-access-djv4b\") pod \"maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:53.745416 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.745327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-maas-api-tls\") pod \"maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:53.846119 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.846086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:53.846304 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.846130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djv4b\" (UniqueName: \"kubernetes.io/projected/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-kube-api-access-djv4b\") pod \"maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:53.846304 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.846218    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-maas-api-tls\") pod \"maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:53.846435 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:58:53.846321    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-serving-cert: secret "maas-api-e2e-shared-b-92be65-serving-cert" not found
Jul 12 18:58:53.846435 ip-10-0-130-164 kubenswrapper[2562]: E0712 18:58:53.846379    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-maas-api-tls podName:efefc966-7f0c-4ba8-a03e-2cee9d688bf5 nodeName:}" failed. No retries permitted until 2026-07-12 18:58:54.346360356 +0000 UTC m=+2017.229710609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-maas-api-tls") pod "maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk" (UID: "efefc966-7f0c-4ba8-a03e-2cee9d688bf5") : secret "maas-api-e2e-shared-b-92be65-serving-cert" not found
Jul 12 18:58:53.846845 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.846823    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:53.854725 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.854702    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djv4b\" (UniqueName: \"kubernetes.io/projected/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-kube-api-access-djv4b\") pod \"maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:53.982640 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:53.982604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:54.165449 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:54.165415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:54.267243 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:54.267150    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:54.350477 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:54.350412    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-maas-api-tls\") pod \"maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:54.352848 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:54.352820    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-maas-api-tls\") pod \"maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:54.538097 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:54.538001    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:54.667619 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:54.667587    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"]
Jul 12 18:58:54.670590 ip-10-0-130-164 kubenswrapper[2562]: W0712 18:58:54.670561    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefefc966_7f0c_4ba8_a03e_2cee9d688bf5.slice/crio-9a572eb6975e043f7a3b7b965f91c0806fde56ee90981f503f759e6a1c827ddc WatchSource:0}: Error finding container 9a572eb6975e043f7a3b7b965f91c0806fde56ee90981f503f759e6a1c827ddc: Status 404 returned error can't find the container with id 9a572eb6975e043f7a3b7b965f91c0806fde56ee90981f503f759e6a1c827ddc
Jul 12 18:58:55.614047 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:55.614012    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:55.658881 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:55.658847    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk" event={"ID":"efefc966-7f0c-4ba8-a03e-2cee9d688bf5","Type":"ContainerStarted","Data":"d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944"}
Jul 12 18:58:55.659046 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:55.658900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk" event={"ID":"efefc966-7f0c-4ba8-a03e-2cee9d688bf5","Type":"ContainerStarted","Data":"9a572eb6975e043f7a3b7b965f91c0806fde56ee90981f503f759e6a1c827ddc"}
Jul 12 18:58:55.659122 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:55.659042    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:58:55.666626 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:55.666602    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:58:55.675808 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:55.675771    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk" podStartSLOduration=2.385251604 podStartE2EDuration="2.675758083s" totalImagesPullingTime="290.506479ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:58:55.67359623 +0000 UTC m=+2018.556953059" watchObservedRunningTime="2026-07-12 18:58:55.675758083 +0000 UTC m=+2018.559108357"
Jul 12 18:58:57.811693 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:58:57.811651    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:59:01.669035 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:59:01.669004    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 18:59:06.789117 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:59:06.789079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:59:06.889542 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:59:06.889509    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:59:14.981930 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:59:14.981895    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 18:59:15.081937 ip-10-0-130-164 kubenswrapper[2562]: I0712 18:59:15.081893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:00:11.348560 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:00:11.348523    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:00:17.629195 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:00:17.629164    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 19:00:17.638383 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:00:17.638359    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 19:02:02.653132 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:02.653102    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:03.020766 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.020729    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:03.121412 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.121377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:03.180317 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.180285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:03.256649 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.256611    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"]
Jul 12 19:02:03.256892 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.256869    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf" podUID="4f5356ea-c2f9-4b07-8f4f-7508e0f69d44" containerName="maas-api" containerID="cri-o://5726f766f615f3045efb686deb421b39c1c4d07a9ce1073f4c2f3bbb937bc7c9" gracePeriod=30
Jul 12 19:02:03.281068 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.281036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:03.401649 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.401549    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:03.419041 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.419013    2562 generic.go:356] "Generic (PLEG): container finished" podID="4f5356ea-c2f9-4b07-8f4f-7508e0f69d44" containerID="5726f766f615f3045efb686deb421b39c1c4d07a9ce1073f4c2f3bbb937bc7c9" exitCode=0
Jul 12 19:02:03.419207 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.419086    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf" event={"ID":"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44","Type":"ContainerDied","Data":"5726f766f615f3045efb686deb421b39c1c4d07a9ce1073f4c2f3bbb937bc7c9"}
Jul 12 19:02:03.456704 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.456669    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:03.497192 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.497168    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 19:02:03.659473 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.659372    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qzj8\" (UniqueName: \"kubernetes.io/projected/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-kube-api-access-6qzj8\") pod \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") "
Jul 12 19:02:03.659473 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.659406    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-maas-api-tls\") pod \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") "
Jul 12 19:02:03.659921 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.659597    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-openshift-service-ca\") pod \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\" (UID: \"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44\") "
Jul 12 19:02:03.660006 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.659980    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f5356ea-c2f9-4b07-8f4f-7508e0f69d44" (UID: "4f5356ea-c2f9-4b07-8f4f-7508e0f69d44"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 19:02:03.661502 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.661480    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f5356ea-c2f9-4b07-8f4f-7508e0f69d44" (UID: "4f5356ea-c2f9-4b07-8f4f-7508e0f69d44"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 19:02:03.661587 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.661568    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-kube-api-access-6qzj8" (OuterVolumeSpecName: "kube-api-access-6qzj8") pod "4f5356ea-c2f9-4b07-8f4f-7508e0f69d44" (UID: "4f5356ea-c2f9-4b07-8f4f-7508e0f69d44"). InnerVolumeSpecName "kube-api-access-6qzj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 19:02:03.760622 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.760583    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 19:02:03.760622 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.760617    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qzj8\" (UniqueName: \"kubernetes.io/projected/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-kube-api-access-6qzj8\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 19:02:03.760622 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:03.760630    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 19:02:04.424932 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:04.424898    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf" event={"ID":"4f5356ea-c2f9-4b07-8f4f-7508e0f69d44","Type":"ContainerDied","Data":"e7a815716d14a64875f8d05b9029b004b1fcae8a362dae496fe1e5b93f24de2f"}
Jul 12 19:02:04.425132 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:04.424955    2562 scope.go:117] "RemoveContainer" containerID="5726f766f615f3045efb686deb421b39c1c4d07a9ce1073f4c2f3bbb937bc7c9"
Jul 12 19:02:04.425132 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:04.424911    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"
Jul 12 19:02:04.445057 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:04.445026    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"]
Jul 12 19:02:04.448706 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:04.448679    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e0bc69-779698d484-gmxxf"]
Jul 12 19:02:05.602515 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:05.602485    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f5356ea-c2f9-4b07-8f4f-7508e0f69d44" path="/var/lib/kubelet/pods/4f5356ea-c2f9-4b07-8f4f-7508e0f69d44/volumes"
Jul 12 19:02:08.615262 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:08.615228    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:08.670137 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:08.670107    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:08.869533 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:08.869423    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:09.243917 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:09.243880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:09.646292 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:09.646157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:09.746934 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:09.746889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:09.847096 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:09.847063    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:09.929450 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:09.929349    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"]
Jul 12 19:02:09.929668 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:09.929639    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk" podUID="efefc966-7f0c-4ba8-a03e-2cee9d688bf5" containerName="maas-api" containerID="cri-o://d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944" gracePeriod=30
Jul 12 19:02:09.958932 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:09.958884    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:10.170554 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.170531    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 19:02:10.319041 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.319011    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-maas-api-tls\") pod \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") "
Jul 12 19:02:10.319227 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.319049    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-openshift-service-ca\") pod \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") "
Jul 12 19:02:10.319227 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.319079    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djv4b\" (UniqueName: \"kubernetes.io/projected/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-kube-api-access-djv4b\") pod \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\" (UID: \"efefc966-7f0c-4ba8-a03e-2cee9d688bf5\") "
Jul 12 19:02:10.319444 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.319417    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efefc966-7f0c-4ba8-a03e-2cee9d688bf5" (UID: "efefc966-7f0c-4ba8-a03e-2cee9d688bf5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 19:02:10.321109 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.321079    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-kube-api-access-djv4b" (OuterVolumeSpecName: "kube-api-access-djv4b") pod "efefc966-7f0c-4ba8-a03e-2cee9d688bf5" (UID: "efefc966-7f0c-4ba8-a03e-2cee9d688bf5"). InnerVolumeSpecName "kube-api-access-djv4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 19:02:10.321215 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.321110    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efefc966-7f0c-4ba8-a03e-2cee9d688bf5" (UID: "efefc966-7f0c-4ba8-a03e-2cee9d688bf5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 19:02:10.420426 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.420392    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-maas-api-tls\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 19:02:10.420426 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.420419    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-openshift-service-ca\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 19:02:10.420426 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.420432    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djv4b\" (UniqueName: \"kubernetes.io/projected/efefc966-7f0c-4ba8-a03e-2cee9d688bf5-kube-api-access-djv4b\") on node \"ip-10-0-130-164.ec2.internal\" DevicePath \"\""
Jul 12 19:02:10.454400 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.454367    2562 generic.go:356] "Generic (PLEG): container finished" podID="efefc966-7f0c-4ba8-a03e-2cee9d688bf5" containerID="d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944" exitCode=0
Jul 12 19:02:10.454599 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.454412    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"
Jul 12 19:02:10.454599 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.454430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk" event={"ID":"efefc966-7f0c-4ba8-a03e-2cee9d688bf5","Type":"ContainerDied","Data":"d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944"}
Jul 12 19:02:10.454599 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.454493    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk" event={"ID":"efefc966-7f0c-4ba8-a03e-2cee9d688bf5","Type":"ContainerDied","Data":"9a572eb6975e043f7a3b7b965f91c0806fde56ee90981f503f759e6a1c827ddc"}
Jul 12 19:02:10.454599 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.454516    2562 scope.go:117] "RemoveContainer" containerID="d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944"
Jul 12 19:02:10.463044 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.463025    2562 scope.go:117] "RemoveContainer" containerID="d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944"
Jul 12 19:02:10.463332 ip-10-0-130-164 kubenswrapper[2562]: E0712 19:02:10.463315    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944\": container with ID starting with d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944 not found: ID does not exist" containerID="d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944"
Jul 12 19:02:10.463383 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.463342    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944"} err="failed to get container status \"d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944\": rpc error: code = NotFound desc = could not find container \"d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944\": container with ID starting with d8693a6ebcb4bf2a18c8c1ee05470422262ca0224f2b1506d5b5cf75c7621944 not found: ID does not exist"
Jul 12 19:02:10.473493 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.473448    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"]
Jul 12 19:02:10.475883 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:10.475857    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-92be65-86db598f5c-gn9rk"]
Jul 12 19:02:11.602280 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:11.602246    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efefc966-7f0c-4ba8-a03e-2cee9d688bf5" path="/var/lib/kubelet/pods/efefc966-7f0c-4ba8-a03e-2cee9d688bf5/volumes"
Jul 12 19:02:15.335758 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:15.335721    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:15.481103 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:15.481069    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:15.621953 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:15.621875    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:16.387031 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:16.386989    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b6bb9c74c-lblfj_49bd55f6-05cf-473a-baa4-bd4ae76a9ecb/authorino/0.log"
Jul 12 19:02:21.628890 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:21.628863    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f88dd4c8-qpzh2_610c69ab-8066-4ed3-9b87-86c9a36b7dcf/copy-manifests/0.log"
Jul 12 19:02:21.640437 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:21.640412    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f88dd4c8-qpzh2_610c69ab-8066-4ed3-9b87-86c9a36b7dcf/manager/0.log"
Jul 12 19:02:22.849870 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:22.849839    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b6bb9c74c-lblfj_49bd55f6-05cf-473a-baa4-bd4ae76a9ecb/authorino/0.log"
Jul 12 19:02:23.174663 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:23.174588    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-z82qs_d903caac-eeb0-479e-80b6-ad2b8b5b069b/kuadrant-console-plugin/0.log"
Jul 12 19:02:24.310661 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:24.310636    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd95989b6-klbnw_4a264575-50da-40c7-a0e5-4133810b085d/kube-auth-proxy/0.log"
Jul 12 19:02:25.447616 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:25.447587    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtbj9b_a5f99b35-910c-4ff6-9f06-7a8a5747a49b/main/0.log"
Jul 12 19:02:25.556220 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:25.556195    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-q4cbr_1219589d-93c9-4ed4-bea9-e4b717156ffc/main/0.log"
Jul 12 19:02:25.662962 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:25.662938    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-cwjb9_29b9beae-472d-416e-aba7-aec223326920/main/0.log"
Jul 12 19:02:29.456897 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:29.456868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:29.524392 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:29.524362    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log" limit=1024
Jul 12 19:02:32.407842 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:32.407806    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4kw4m_adc887e4-8e0a-4ed3-9978-792e0a1573bb/global-pull-secret-syncer/0.log"
Jul 12 19:02:32.528850 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:32.528825    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b988b_5183a6f1-0901-4208-b85a-8fab51402e3e/konnectivity-agent/0.log"
Jul 12 19:02:32.615409 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:32.615381    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-164.ec2.internal_2a48c8d39a9521700ee4336712fb4822/haproxy/0.log"
Jul 12 19:02:37.575083 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:37.575055    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b6bb9c74c-lblfj_49bd55f6-05cf-473a-baa4-bd4ae76a9ecb/authorino/0.log"
Jul 12 19:02:37.659428 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:37.659401    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-z82qs_d903caac-eeb0-479e-80b6-ad2b8b5b069b/kuadrant-console-plugin/0.log"
Jul 12 19:02:40.023315 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:40.023248    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f56gm_a28fc4fa-db9a-4279-aa00-fedb855f66a3/node-exporter/0.log"
Jul 12 19:02:40.070035 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:40.070013    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f56gm_a28fc4fa-db9a-4279-aa00-fedb855f66a3/kube-rbac-proxy/0.log"
Jul 12 19:02:40.089236 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:40.089213    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f56gm_a28fc4fa-db9a-4279-aa00-fedb855f66a3/init-textfile/0.log"
Jul 12 19:02:43.251445 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.251411    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"]
Jul 12 19:02:43.251844 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.251695    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efefc966-7f0c-4ba8-a03e-2cee9d688bf5" containerName="maas-api"
Jul 12 19:02:43.251844 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.251705    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="efefc966-7f0c-4ba8-a03e-2cee9d688bf5" containerName="maas-api"
Jul 12 19:02:43.251844 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.251722    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f5356ea-c2f9-4b07-8f4f-7508e0f69d44" containerName="maas-api"
Jul 12 19:02:43.251844 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.251728    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f5356ea-c2f9-4b07-8f4f-7508e0f69d44" containerName="maas-api"
Jul 12 19:02:43.251844 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.251795    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="efefc966-7f0c-4ba8-a03e-2cee9d688bf5" containerName="maas-api"
Jul 12 19:02:43.251844 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.251804    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f5356ea-c2f9-4b07-8f4f-7508e0f69d44" containerName="maas-api"
Jul 12 19:02:43.254587 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.254569    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.256830 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.256811    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pxs8x\"/\"kube-root-ca.crt\""
Jul 12 19:02:43.256940 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.256892    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pxs8x\"/\"default-dockercfg-6wkmn\""
Jul 12 19:02:43.257619 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.257603    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pxs8x\"/\"openshift-service-ca.crt\""
Jul 12 19:02:43.263740 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.263719    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"]
Jul 12 19:02:43.394118 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.394082    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-lib-modules\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.394118 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.394131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4cwh\" (UniqueName: \"kubernetes.io/projected/5f842629-9125-4e23-89fa-4827e5436e0a-kube-api-access-r4cwh\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.394348 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.394208    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-proc\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.394348 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.394279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-sys\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.394348 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.394334    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-podres\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.495645 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.495599    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-proc\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.495773 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.495699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-sys\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.495773 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.495713    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-proc\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.495773 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.495727    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-podres\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.495773 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.495755    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-sys\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.495773 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.495760    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-lib-modules\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.496026 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.495790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4cwh\" (UniqueName: \"kubernetes.io/projected/5f842629-9125-4e23-89fa-4827e5436e0a-kube-api-access-r4cwh\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.496026 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.495826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-podres\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.496026 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.495908    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f842629-9125-4e23-89fa-4827e5436e0a-lib-modules\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.504811 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.504749    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4cwh\" (UniqueName: \"kubernetes.io/projected/5f842629-9125-4e23-89fa-4827e5436e0a-kube-api-access-r4cwh\") pod \"perf-node-gather-daemonset-d9lb5\" (UID: \"5f842629-9125-4e23-89fa-4827e5436e0a\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.564768 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.564734    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:43.692275 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:43.692227    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"]
Jul 12 19:02:43.694486 ip-10-0-130-164 kubenswrapper[2562]: W0712 19:02:43.694446    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5f842629_9125_4e23_89fa_4827e5436e0a.slice/crio-47e5b34ffbbf2a98521768953ec822a2238d356f52cff4c2f7a718c5a1cd7634 WatchSource:0}: Error finding container 47e5b34ffbbf2a98521768953ec822a2238d356f52cff4c2f7a718c5a1cd7634: Status 404 returned error can't find the container with id 47e5b34ffbbf2a98521768953ec822a2238d356f52cff4c2f7a718c5a1cd7634
Jul 12 19:02:44.591637 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:44.591604    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5" event={"ID":"5f842629-9125-4e23-89fa-4827e5436e0a","Type":"ContainerStarted","Data":"72bcce3525e65788a1241b0a7dd040d1456fee900ccd70bb37e1c5bb01ee8acd"}
Jul 12 19:02:44.592018 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:44.591654    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5" event={"ID":"5f842629-9125-4e23-89fa-4827e5436e0a","Type":"ContainerStarted","Data":"47e5b34ffbbf2a98521768953ec822a2238d356f52cff4c2f7a718c5a1cd7634"}
Jul 12 19:02:44.592018 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:44.591722    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:44.605938 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:44.605888    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5" podStartSLOduration=1.605876502 podStartE2EDuration="1.605876502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 19:02:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:02:44.605525905 +0000 UTC m=+2247.488876180" watchObservedRunningTime="2026-07-12 19:02:44.605876502 +0000 UTC m=+2247.489226790"
Jul 12 19:02:45.069220 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:45.069170    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h8jls_cee6a773-686d-4ef4-a904-110a2826bce6/dns/0.log"
Jul 12 19:02:45.089422 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:45.089402    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h8jls_cee6a773-686d-4ef4-a904-110a2826bce6/kube-rbac-proxy/0.log"
Jul 12 19:02:45.206573 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:45.206541    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f4xh5_b098ac18-b58d-4a14-af63-7b1973beb36b/dns-node-resolver/0.log"
Jul 12 19:02:45.803955 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:45.803925    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-677fddf46f-2hds5_6b5204ff-2f07-435b-a169-2210ca8a2b6b/registry/0.log"
Jul 12 19:02:45.844983 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:45.844961    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rxs9c_6f73fd2b-9f51-44df-98cb-672c606fc32e/node-ca/0.log"
Jul 12 19:02:46.924727 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:46.924698    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd95989b6-klbnw_4a264575-50da-40c7-a0e5-4133810b085d/kube-auth-proxy/0.log"
Jul 12 19:02:47.799117 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:47.799092    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7msr2_2be4bca4-2772-4c13-a24b-eebc9d61cc7d/serve-healthcheck-canary/0.log"
Jul 12 19:02:48.621226 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:48.621196    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-znnz5_8476a109-14a2-44ba-b1b7-69d80207150f/kube-rbac-proxy/0.log"
Jul 12 19:02:48.644727 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:48.644700    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-znnz5_8476a109-14a2-44ba-b1b7-69d80207150f/exporter/0.log"
Jul 12 19:02:48.668898 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:48.668874    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-znnz5_8476a109-14a2-44ba-b1b7-69d80207150f/extractor/0.log"
Jul 12 19:02:50.605411 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:50.605383    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-d9lb5"
Jul 12 19:02:51.359796 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:51.359759    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f88dd4c8-qpzh2_610c69ab-8066-4ed3-9b87-86c9a36b7dcf/manager/0.log"
Jul 12 19:02:51.375276 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:51.375253    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f88dd4c8-qpzh2_610c69ab-8066-4ed3-9b87-86c9a36b7dcf/copy-manifests/0.log"
Jul 12 19:02:52.707560 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:52.707478    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7f877c4f85-4qbqm_a885e50f-a14d-420c-b95f-f6a01d5b2f84/manager/0.log"
Jul 12 19:02:54.422250 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:54.422223    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd95989b6-klbnw_4a264575-50da-40c7-a0e5-4133810b085d/kube-auth-proxy/0.log"
Jul 12 19:02:56.084896 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:56.084869    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd95989b6-klbnw_4a264575-50da-40c7-a0e5-4133810b085d/kube-auth-proxy/0.log"
Jul 12 19:02:58.225840 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:58.225812    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-6bh8p_a23e3158-dd56-487c-bb12-3f7c5fec467d/migrator/0.log"
Jul 12 19:02:58.246435 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:58.246405    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-6bh8p_a23e3158-dd56-487c-bb12-3f7c5fec467d/graceful-termination/0.log"
Jul 12 19:02:59.715577 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:59.715552    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gx47v_dec7786a-3b72-4a67-9aea-8d1030b7cbab/kube-multus-additional-cni-plugins/0.log"
Jul 12 19:02:59.735105 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:59.735076    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gx47v_dec7786a-3b72-4a67-9aea-8d1030b7cbab/egress-router-binary-copy/0.log"
Jul 12 19:02:59.755214 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:59.755187    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gx47v_dec7786a-3b72-4a67-9aea-8d1030b7cbab/cni-plugins/0.log"
Jul 12 19:02:59.774766 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:59.774745    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gx47v_dec7786a-3b72-4a67-9aea-8d1030b7cbab/bond-cni-plugin/0.log"
Jul 12 19:02:59.794310 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:59.794289    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gx47v_dec7786a-3b72-4a67-9aea-8d1030b7cbab/routeoverride-cni/0.log"
Jul 12 19:02:59.816193 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:59.816171    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gx47v_dec7786a-3b72-4a67-9aea-8d1030b7cbab/whereabouts-cni-bincopy/0.log"
Jul 12 19:02:59.836087 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:02:59.836065    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gx47v_dec7786a-3b72-4a67-9aea-8d1030b7cbab/whereabouts-cni/0.log"
Jul 12 19:03:00.196312 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:03:00.196281    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q9sm4_6ff40a73-e740-4c46-9240-d5dba77aeefc/kube-multus/0.log"
Jul 12 19:03:00.253974 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:03:00.253947    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cbzdr_77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0/network-metrics-daemon/0.log"
Jul 12 19:03:00.272008 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:03:00.271970    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cbzdr_77a776d1-4aa1-4bdc-86f5-b8bd2097e5e0/kube-rbac-proxy/0.log"
Jul 12 19:03:01.428763 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:03:01.428729    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-controller/0.log"
Jul 12 19:03:01.445375 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:03:01.445355    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/1.log"
Jul 12 19:03:01.445534 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:03:01.445420    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/ovn-acl-logging/0.log"
Jul 12 19:03:01.462238 ip-10-0-130-164 kubenswrapper[2562]: I0712 19:03:01.462215    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p8p68_c3e2aa5d-4161-4eef-aaa4-5ce5655d4530/kube-rbac-proxy-node/0.log"