Jul 20 15:50:34.008355 ip-10-0-134-220 systemd[1]: Starting Kubernetes Kubelet...
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.464094    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473381    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473396    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473399    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473403    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473405    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473408    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473411    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:50:34.497977 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473413    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473416    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473419    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473422    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473426    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473429    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473434    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473446    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473450    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473453    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473456    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473458    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473461    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473464    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473466    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473469    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473473    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473475    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473478    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473480    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:50:34.783903 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473483    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:50:34.543635 ip-10-0-134-220 systemd[1]: Started Kubernetes Kubelet.
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473486    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473488    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473491    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473495    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473506    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473510    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473514    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473517    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473520    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473523    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473526    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473529    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473532    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473534    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473537    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473540    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473544    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473546    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473549    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:50:34.797426 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473553    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473557    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473560    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473563    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473566    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473568    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473571    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473574    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473576    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473579    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473581    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473584    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473586    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473589    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473591    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473594    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473596    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473599    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473602    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:50:34.798829 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473604    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473607    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473609    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473612    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473615    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473618    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473620    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473623    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473626    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473629    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473631    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473634    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473636    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473639    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473641    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473644    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473646    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473649    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473651    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:50:34.799847 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473654    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473657    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473659    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473662    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473664    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473667    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473670    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473672    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473675    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473677    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473680    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473682    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473685    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.473687    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474425    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474432    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474435    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474439    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474487    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474537    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:50:34.800885 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474541    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474544    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474546    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474550    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474553    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474555    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474558    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474561    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474564    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474566    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474569    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474572    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474578    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474581    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474584    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474586    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474589    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474593    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474596    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474599    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:50:34.801898 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.474602    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.475901    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.475924    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476068    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476081    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476086    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476090    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476093    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476096    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476102    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476105    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476108    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476112    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476114    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476117    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476120    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476123    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476126    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476129    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:50:34.802942 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476132    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476135    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476138    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476140    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476145    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476148    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476151    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476154    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476157    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476160    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476163    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476166    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476169    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476171    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476174    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476177    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476180    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476182    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476185    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476188    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:50:34.803974 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476191    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476195    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476200    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476203    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476206    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476208    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476211    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476214    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476216    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476220    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476223    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476226    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476229    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476232    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476234    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476237    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476240    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476242    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476245    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476247    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:50:34.805015 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476250    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476252    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476255    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476257    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476260    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476263    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476265    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476268    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476270    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476272    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476275    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476278    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476280    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476283    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476685    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476690    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476693    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476698    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476702    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:50:34.806037 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476705    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476708    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476711    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476713    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476716    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476718    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476721    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476723    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476726    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476728    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476731    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476733    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476736    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476738    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476741    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476743    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476746    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476748    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476751    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476753    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:50:34.807112 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476755    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476758    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476761    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476763    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476766    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476768    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476770    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476778    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476780    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476783    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476786    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476788    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476791    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476793    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476796    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476799    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476802    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476804    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476807    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:50:34.808128 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476809    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476813    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476816    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476819    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476822    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476824    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476827    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476830    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476832    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476835    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476838    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476841    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476843    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476845    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476848    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476850    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476853    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476855    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476858    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:50:34.809188 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476860    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476866    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476869    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476872    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476874    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476876    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476879    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476882    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476884    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476887    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476891    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476894    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476897    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476900    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476903    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476906    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476908    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476911    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476913    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:50:34.810242 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476916    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476919    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476922    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476926    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476930    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476932    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476934    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476937    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476939    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476942    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476944    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476947    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476949    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476952    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476956    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476959    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.476962    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477034    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477042    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477068    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477073    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 20 15:50:34.811269 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477077    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477080    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477085    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477089    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477092    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477095    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477099    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477102    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477105    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477108    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477111    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477114    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477116    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477119    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477123    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477126    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477130    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477133    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477136    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477140    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477143    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477146    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477149    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477152    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 20 15:50:34.812249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477155    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477161    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477164    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477166    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477170    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477173    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477176    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477179    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477182    2564 flags.go:64] FLAG: --enable-server="true"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477185    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477190    2564 flags.go:64] FLAG: --event-burst="100"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477193    2564 flags.go:64] FLAG: --event-qps="50"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477196    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477199    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477201    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477205    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477208    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477211    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477214    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477216    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477219    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477222    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477225    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477227    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477230    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 20 15:50:34.813286 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477236    2564 flags.go:64] FLAG: --feature-gates=""
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477241    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477244    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477247    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477250    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477253    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477256    2564 flags.go:64] FLAG: --help="false"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477259    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-134-220.ec2.internal"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477265    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477268    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477271    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477274    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477278    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477281    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477283    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477286    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477289    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477292    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477295    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477298    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477301    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477303    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477306    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477309    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 20 15:50:34.814801 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477312    2564 flags.go:64] FLAG: --lock-file=""
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477314    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477317    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477320    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477326    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477328    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477331    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477334    2564 flags.go:64] FLAG: --logging-format="text"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477336    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477340    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477343    2564 flags.go:64] FLAG: --manifest-url=""
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477346    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477350    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477353    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477357    2564 flags.go:64] FLAG: --max-pods="110"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477360    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477365    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477368    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477371    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477373    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477376    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477379    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477386    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477389    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477392    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 20 15:50:34.815861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477395    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477398    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477403    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477406    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477409    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477412    2564 flags.go:64] FLAG: --port="10250"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477415    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477418    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cb3d8a7680890a5c"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477421    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477424    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477426    2564 flags.go:64] FLAG: --register-node="true"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477430    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477433    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477443    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477446    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477449    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477453    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477457    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477460    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477463    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477466    2564 flags.go:64] FLAG: --runonce="false"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477469    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477472    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477476    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477480    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 20 15:50:34.816954 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477483    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477486    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477489    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477493    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477495    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477498    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477501    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477504    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477507    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477510    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477513    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477518    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477521    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477523    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477527    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477530    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477533    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477536    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477538    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477541    2564 flags.go:64] FLAG: --v="2"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477545    2564 flags.go:64] FLAG: --version="false"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477549    2564 flags.go:64] FLAG: --vmodule=""
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477552    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477555    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477670    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:50:34.822155 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477674    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477677    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477680    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477682    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477685    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477689    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477693    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477695    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477698    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477700    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477703    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477705    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477708    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477711    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477713    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477716    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477718    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477721    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477724    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477726    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:50:34.823628 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477729    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477731    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477734    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477737    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477739    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477742    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477744    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477747    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477750    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477753    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477755    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477758    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477760    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477764    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477766    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477769    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477771    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477775    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477781    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477783    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:50:34.932696 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477786    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477788    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477791    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477794    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477796    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477799    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477801    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477804    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477806    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477809    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477811    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477814    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477816    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477819    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477821    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477825    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477829    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477832    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477834    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:50:34.933725 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477837    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477840    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477843    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477845    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477848    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477850    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477853    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477856    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477859    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477861    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477866    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477869    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477872    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477875    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477877    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477880    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477883    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477885    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477888    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:50:34.934743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477892    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477895    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477898    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477901    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477905    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477908    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477910    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477914    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477916    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477919    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477922    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477924    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477928    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477931    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477934    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477937    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477940    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477942    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477945    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:50:34.935756 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.477947    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.477953    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.489147    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.489167    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.490161    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.490169    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490229    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490234    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490239    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490243    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490246    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490248    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490251    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490254    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:50:34.936749 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490257    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490259    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490262    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490264    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490267    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490270    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490273    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490275    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490278    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490281    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490284    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490287    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490289    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490292    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490295    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490297    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490317    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490325    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490328    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490331    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:50:34.937743 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490334    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490338    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490341    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490343    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490346    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490349    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490351    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490354    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490356    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490359    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490361    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490364    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490367    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490369    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490372    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490374    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490379    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490382    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490385    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:50:34.938766 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490388    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490391    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490393    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490396    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490399    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490401    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490404    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490407    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490409    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490411    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490416    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490418    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490421    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490424    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490426    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490429    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490432    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490435    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490438    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:50:34.939793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490440    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490443    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490445    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490448    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490450    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490453    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490455    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490458    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490460    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490463    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490466    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490468    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490472    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490475    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490478    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490481    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490484    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490486    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490489    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490492    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:50:34.940799 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490494    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490497    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490500    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490504    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490507    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490509    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490512    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490514    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490517    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490520    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490522    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490525    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490527    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.490532    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490649    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:50:34.941835 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490653    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490657    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490661    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490664    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490666    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490670    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490672    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490675    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490677    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490680    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490683    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490686    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490688    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490691    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490693    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490696    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490698    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490701    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490703    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490705    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:50:34.942802 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490710    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490714    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490718    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490721    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490724    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490727    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490730    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490733    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490736    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490738    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490740    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490743    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490746    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490748    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490751    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490753    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490756    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490758    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490761    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490763    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:50:34.943830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490765    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490768    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490771    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490773    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490775    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490778    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490781    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490783    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490786    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490788    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490790    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490793    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490797    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490801    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490804    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490806    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490809    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490812    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490814    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:50:34.944830 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490817    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490820    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490822    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490825    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490828    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490830    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490833    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490836    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490838    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490841    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490843    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490846    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490848    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490851    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490854    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490856    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490859    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490862    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490864    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:50:34.945854 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490867    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490869    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490872    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490874    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490878    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490881    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490885    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490889    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490892    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490895    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490898    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490900    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490903    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490905    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490908    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490911    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490914    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490917    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490919    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:50:34.946855 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:34.490921    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.490926    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.491073    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.493452    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.494849    2564 server.go:1019] "Starting client certificate rotation"
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.494955    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.494991    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.497611    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.498809    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.506998    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.509460    2564 log.go:25] "Validated CRI v1 image API"
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.513449    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 20 15:50:34.947867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.516386    2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9e4bacea-4902-4ece-89b2-477238671a8a:/dev/nvme0n1p4 c5de918a-14c6-40c1-9a79-23e6d66cb440:/dev/nvme0n1p3]
Jul 20 15:50:34.948827 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.516404    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 20 15:50:34.949305 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.521360    2564 manager.go:216] Machine: {Timestamp:2026-07-20 15:50:34.52000908 +0000 UTC m=+0.368913363 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099955 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208255cceaf63ea0ba19b0d40f1f0f SystemUUID:ec208255-ccea-f63e-a0ba-19b0d40f1f0f BootID:61c0cb12-014d-4a96-9220-9799c11aaa62 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ad:ed:f9:4a:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ad:ed:f9:4a:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:7b:0e:1a:f3:23 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 20 15:50:34.949305 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.521470    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 20 15:50:34.949305 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.521551    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 20 15:50:34.949305 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.524790    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 20 15:50:34.949305 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.524816    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-220.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 20 15:50:34.949305 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.526086    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 20 15:50:34.949305 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.526098    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 20 15:50:34.949305 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.526105    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.526974    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.528233    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.531350    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.531899    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.532010    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.532036    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.532058    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.532070    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.532081    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.532093    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.532457    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.532476    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.534754    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.536536    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.536564    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538240    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538256    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538263    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538269    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538277    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538283    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538291    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538297    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538305    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538311    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538319    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.538325    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.539093    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.539103    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.542425    2564 server.go:1268] "Started kubelet"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.542559    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.542542    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.542785    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.543809    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.544218    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.544469    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.544592    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-220.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.544672    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-220.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 20 15:50:34.949825 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.547801    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.549423    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j8cwr"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.550741    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.550835    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.550739    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.551438    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.551453    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.551440    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.551609    2564 factory.go:55] Registering systemd factory
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.551625    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.551628    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.551638    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.551900    2564 factory.go:153] Registering CRI-O factory
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.551915    2564 factory.go:223] Registration of the crio container factory successfully
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.551964    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.552035    2564 factory.go:103] Registering Raw factory
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.552071    2564 manager.go:1196] Started watching for new ooms in manager
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.552449    2564 manager.go:349] Starting recovery of all containers
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.551674    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-220.ec2.internal.18c40a42d7559973  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-220.ec2.internal,UID:ip-10-0-134-220.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-220.ec2.internal,},FirstTimestamp:2026-07-20 15:50:34.542389619 +0000 UTC m=+0.391293905,LastTimestamp:2026-07-20 15:50:34.542389619 +0000 UTC m=+0.391293905,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-220.ec2.internal,}"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.557841    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j8cwr"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.558119    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.558148    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-220.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.564937    2564 manager.go:354] Recovery completed
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.569458    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.571993    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.572018    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.572028    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.572468    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.572479    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.572497    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.572499    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.573310    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:50:34.951812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.573338    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.573353    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.574536    2564 policy_none.go:49] "None policy: Start"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.574554    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.574567    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.574512    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-220.ec2.internal.18c40a42d9198219  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-220.ec2.internal,UID:ip-10-0-134-220.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-220.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-220.ec2.internal,},FirstTimestamp:2026-07-20 15:50:34.572005913 +0000 UTC m=+0.420910197,LastTimestamp:2026-07-20 15:50:34.572005913 +0000 UTC m=+0.420910197,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-220.ec2.internal,}"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.576510    2564 policy_none.go:47] "Start"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.642762    2564 manager.go:337] "Starting Device Plugin manager"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.642804    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.642818    2564 server.go:87] "Starting device plugin registration server"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.643256    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.643268    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.643358    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.643459    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.643469    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.644014    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.644088    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.667664    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.668950    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.668978    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.669006    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.669059    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.671582    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.743939    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.744980    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.745013    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.745027    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.745071    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-220.ec2.internal"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.753642    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-220.ec2.internal"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.753691    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-220.ec2.internal\": node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.769637    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.769418    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal"]
Jul 20 15:50:34.957396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.770387    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.771343    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.771380    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.771396    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.773515    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.773631    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.773661    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.774376    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.774398    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.774412    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.774421    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.774430    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.774435    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.776434    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.776454    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.777152    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.777177    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.777190    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.852772    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cd6f877677494bfec658ee6d0ff608a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"6cd6f877677494bfec658ee6d0ff608a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.852802    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cd6f877677494bfec658ee6d0ff608a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"6cd6f877677494bfec658ee6d0ff608a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.852820    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c2bd98933375e305b9c32417ca912b54-config\") pod \"kube-apiserver-proxy-ip-10-0-134-220.ec2.internal\" (UID: \"c2bd98933375e305b9c32417ca912b54\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.869901    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.953143    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cd6f877677494bfec658ee6d0ff608a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"6cd6f877677494bfec658ee6d0ff608a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:34.958953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.953177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c2bd98933375e305b9c32417ca912b54-config\") pod \"kube-apiserver-proxy-ip-10-0-134-220.ec2.internal\" (UID: \"c2bd98933375e305b9c32417ca912b54\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:35.069561 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.953195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cd6f877677494bfec658ee6d0ff608a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"6cd6f877677494bfec658ee6d0ff608a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:35.069561 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.953246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cd6f877677494bfec658ee6d0ff608a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"6cd6f877677494bfec658ee6d0ff608a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:35.069561 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.953246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cd6f877677494bfec658ee6d0ff608a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"6cd6f877677494bfec658ee6d0ff608a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:35.069561 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:34.953255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c2bd98933375e305b9c32417ca912b54-config\") pod \"kube-apiserver-proxy-ip-10-0-134-220.ec2.internal\" (UID: \"c2bd98933375e305b9c32417ca912b54\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:35.069561 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:34.970247    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:35.069561 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:35.023423    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-220.ec2.internal\" not found" node="ip-10-0-134-220.ec2.internal"
Jul 20 15:50:35.069561 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:35.026129    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:35.069561 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:35.026700    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-220.ec2.internal\" not found" node="ip-10-0-134-220.ec2.internal"
Jul 20 15:50:35.069561 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:35.029363    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:35.071481 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:35.070659    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:35.195160 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:35.171090    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:35.301149 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:35.271616    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:35.417356 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:35.367992    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:50:35.417356 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:35.372284    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:35.536140 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:35.472802    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:35.536140 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:35.493954    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 20 15:50:35.536140 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:35.494167    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 15:50:35.536140 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:35.494169    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 15:50:35.647597 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:35.563873    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 15:45:35 +0000 UTC" deadline="2028-01-21 17:14:40.006035781 +0000 UTC"
Jul 20 15:50:35.647597 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:35.563903    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13201h24m4.44213542s"
Jul 20 15:50:35.647597 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:35.573007    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:36.951859 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:35.673672    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found"
Jul 20 15:50:36.951859 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:36.911784    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:50:36.951859 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:36.945324    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:37.057805 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:36.957415    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 15:50:37.057805 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:36.958466    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal"
Jul 20 15:50:37.057805 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:36.965802    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 15:50:37.057805 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.028853    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:50:37.675198 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.654990    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.726814    2564 apiserver.go:52] "Watching apiserver"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.732118    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.733460    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-t4wh7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl","openshift-dns/node-resolver-4lbp5","openshift-image-registry/node-ca-dvh7c","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal","openshift-multus/multus-additional-cni-plugins-8p6lq","openshift-multus/multus-t597z","openshift-ovn-kubernetes/ovnkube-node-kxp2n","kube-system/konnectivity-agent-99khn","kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal","openshift-cluster-node-tuning-operator/tuned-w5bhp","openshift-multus/network-metrics-daemon-lplvs","openshift-network-diagnostics/network-check-target-hggvm","openshift-network-operator/iptables-alerter-kt49n"]
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.745916    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.754975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-system-cni-dir\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.755001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-binary-copy\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.755018    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.755034    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.755051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b69s\" (UniqueName: \"kubernetes.io/projected/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-kube-api-access-5b69s\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.755074    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cnibin\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.755109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-os-release\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.755129    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.756558    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.775393    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qmw7x"
Jul 20 15:50:37.787508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.782986    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qmw7x"
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-os-release\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-system-cni-dir\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856301    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-os-release\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-binary-copy\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856377    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.856400    2564 configmap.go:193] Couldn't get configMap openshift-multus/cni-copy-resources: object "openshift-multus"/"cni-copy-resources" not registered
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856411    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-system-cni-dir\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.856427    2564 configmap.go:193] Couldn't get configMap openshift-multus/whereabouts-flatfile-config: object "openshift-multus"/"whereabouts-flatfile-config" not registered
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.856437    2564 configmap.go:193] Couldn't get configMap openshift-multus/default-cni-sysctl-allowlist: object "openshift-multus"/"default-cni-sysctl-allowlist" not registered
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.856485    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-binary-copy podName:a46171f5-26e9-4fda-9f5d-6e1ef9a0995e nodeName:}" failed. No retries permitted until 2026-07-20 15:50:38.356443814 +0000 UTC m=+3.011707089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-binary-copy") pod "multus-additional-cni-plugins-8p6lq" (UID: "a46171f5-26e9-4fda-9f5d-6e1ef9a0995e") : object "openshift-multus"/"cni-copy-resources" not registered
Jul 20 15:50:37.897749 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.856499    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-whereabouts-flatfile-configmap podName:a46171f5-26e9-4fda-9f5d-6e1ef9a0995e nodeName:}" failed. No retries permitted until 2026-07-20 15:50:38.356492431 +0000 UTC m=+3.011755688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "whereabouts-flatfile-configmap" (UniqueName: "kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-whereabouts-flatfile-configmap") pod "multus-additional-cni-plugins-8p6lq" (UID: "a46171f5-26e9-4fda-9f5d-6e1ef9a0995e") : object "openshift-multus"/"whereabouts-flatfile-config" not registered
Jul 20 15:50:37.899386 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856511    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b69s\" (UniqueName: \"kubernetes.io/projected/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-kube-api-access-5b69s\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.899386 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856540    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cnibin\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.899386 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.856556    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-sysctl-allowlist podName:a46171f5-26e9-4fda-9f5d-6e1ef9a0995e nodeName:}" failed. No retries permitted until 2026-07-20 15:50:38.356534827 +0000 UTC m=+3.011798097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cni-sysctl-allowlist" (UniqueName: "kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-sysctl-allowlist") pod "multus-additional-cni-plugins-8p6lq" (UID: "a46171f5-26e9-4fda-9f5d-6e1ef9a0995e") : object "openshift-multus"/"default-cni-sysctl-allowlist" not registered
Jul 20 15:50:37.899386 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.856578    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cnibin\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.899386 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.858294    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.899386 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.862681    2564 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: object "openshift-multus"/"kube-root-ca.crt" not registered
Jul 20 15:50:37.899386 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.862711    2564 projected.go:291] Couldn't get configMap openshift-multus/openshift-service-ca.crt: object "openshift-multus"/"openshift-service-ca.crt" not registered
Jul 20 15:50:37.899386 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.862721    2564 projected.go:196] Error preparing data for projected volume kube-api-access-5b69s for pod openshift-multus/multus-additional-cni-plugins-8p6lq: [object "openshift-multus"/"kube-root-ca.crt" not registered, object "openshift-multus"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:37.899386 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.862771    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-kube-api-access-5b69s podName:a46171f5-26e9-4fda-9f5d-6e1ef9a0995e nodeName:}" failed. No retries permitted until 2026-07-20 15:50:38.36275587 +0000 UTC m=+3.018019137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5b69s" (UniqueName: "kubernetes.io/projected/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-kube-api-access-5b69s") pod "multus-additional-cni-plugins-8p6lq" (UID: "a46171f5-26e9-4fda-9f5d-6e1ef9a0995e") : [object "openshift-multus"/"kube-root-ca.crt" not registered, object "openshift-multus"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:37.939107 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.939032    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:37.942020 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.941372    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-95ztr\""
Jul 20 15:50:37.942020 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.941716    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 20 15:50:37.942772 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.942725    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 20 15:50:37.943193 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.943037    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 20 15:50:37.943193 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.943096    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 20 15:50:37.943193 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.943115    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 20 15:50:37.943372 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.943300    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:37.944856 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.944836    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 20 15:50:37.945094 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.945068    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 20 15:50:37.945344 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.945323    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:37.945461 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.945433    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9c7cd\""
Jul 20 15:50:37.945565 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.945550    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 20 15:50:37.947609 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.947595    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:37.948135 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.948118    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8x5sk\""
Jul 20 15:50:37.948260 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.948242    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 20 15:50:37.948630 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.948590    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 20 15:50:37.949727 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.949712    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:37.949781 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.949754    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s9vx6\""
Jul 20 15:50:37.949839 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.949770    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 20 15:50:37.949839 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.949713    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 20 15:50:37.949839 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.949778    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:37.949957 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.949786    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 20 15:50:37.952184 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.952166    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t597z"
Jul 20 15:50:37.953874 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.953858    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ctmzv\""
Jul 20 15:50:37.954207 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.954192    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 20 15:50:37.954409 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.954392    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:37.954480 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.954462    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:37.956822 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.956805    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:37.956822 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.956830    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-etc-selinux\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:37.956986 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.956848    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0ab2581-3283-415f-b373-d3eedae55bfa-hosts-file\") pod \"node-resolver-4lbp5\" (UID: \"d0ab2581-3283-415f-b373-d3eedae55bfa\") " pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:37.956986 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.956857    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:37.956986 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.956873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44e5c95e-88e3-41b1-88b1-928f8c7f31ae-host-slash\") pod \"iptables-alerter-kt49n\" (UID: \"44e5c95e-88e3-41b1-88b1-928f8c7f31ae\") " pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:37.956986 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.956959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-registration-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:37.957211 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.956993    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpx5k\" (UniqueName: \"kubernetes.io/projected/d0ab2581-3283-415f-b373-d3eedae55bfa-kube-api-access-mpx5k\") pod \"node-resolver-4lbp5\" (UID: \"d0ab2581-3283-415f-b373-d3eedae55bfa\") " pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:37.957211 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.957020    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d92cc\" (UniqueName: \"kubernetes.io/projected/44e5c95e-88e3-41b1-88b1-928f8c7f31ae-kube-api-access-d92cc\") pod \"iptables-alerter-kt49n\" (UID: \"44e5c95e-88e3-41b1-88b1-928f8c7f31ae\") " pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:37.957211 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.957116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-socket-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:37.957211 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.957155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44e5c95e-88e3-41b1-88b1-928f8c7f31ae-iptables-alerter-script\") pod \"iptables-alerter-kt49n\" (UID: \"44e5c95e-88e3-41b1-88b1-928f8c7f31ae\") " pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:37.957399 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.957215    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-device-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:37.957399 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.957250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-sys-fs\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:37.957399 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.957281    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w2qx\" (UniqueName: \"kubernetes.io/projected/2217f173-6651-4cfb-86d7-68c3843adf07-kube-api-access-7w2qx\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:37.957399 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.957331    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ab2581-3283-415f-b373-d3eedae55bfa-tmp-dir\") pod \"node-resolver-4lbp5\" (UID: \"d0ab2581-3283-415f-b373-d3eedae55bfa\") " pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:37.958922 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.958903    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 20 15:50:37.959016 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.958971    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 20 15:50:37.959125 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.959087    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 20 15:50:37.959259 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.959246    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lwk2q\""
Jul 20 15:50:37.959494 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.959478    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:50:37.959913 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.959896    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 20 15:50:37.960089 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.960041    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 20 15:50:37.960089 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.960084    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 20 15:50:37.961981 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.961965    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 20 15:50:37.961981 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.961979    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-75fhk\""
Jul 20 15:50:37.962114 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.961971    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 20 15:50:37.962114 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.962046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:37.964582 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.964552    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:37.964655 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:37.964624    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:37.965168 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.965151    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hf8bp\""
Jul 20 15:50:37.965241 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.965174    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 20 15:50:37.965241 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.965215    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 20 15:50:37.966899 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.966885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:37.971005 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.970986    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 20 15:50:37.972155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.972139    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 20 15:50:37.972353 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.972328    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nbf4b\""
Jul 20 15:50:37.972437 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:37.972411    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 20 15:50:38.046201 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.046179    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 20 15:50:38.058136 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.058232 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058143    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhckn\" (UniqueName: \"kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn\") pod \"network-check-target-hggvm\" (UID: \"b63c3b63-90d2-45eb-ae0d-8491fa36761c\") " pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:38.058232 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/178ab065-8406-4e3f-8005-b6771517935e-cni-binary-copy\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.058232 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058178    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-env-overrides\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.058232 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058222    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-etc-openvswitch\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.058379 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058286    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl67g\" (UniqueName: \"kubernetes.io/projected/a518917c-09da-43c3-b61e-f17a704fa9a2-kube-api-access-nl67g\") pod \"node-ca-dvh7c\" (UID: \"a518917c-09da-43c3-b61e-f17a704fa9a2\") " pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:38.058379 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058318    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/672c6793-5272-43d1-aa5d-ce5bf204f783-dbus\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:38.058379 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058338    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-kubelet\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.058379 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-run-openvswitch\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.058379 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058369    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-ovnkube-config\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.058548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058399    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-kubernetes\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.058548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058421    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-sysctl-conf\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.058548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058437    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a518917c-09da-43c3-b61e-f17a704fa9a2-host\") pod \"node-ca-dvh7c\" (UID: \"a518917c-09da-43c3-b61e-f17a704fa9a2\") " pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:38.058548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058452    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-slash\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.058548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058475    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/73455279-4b14-4bd5-923a-a23b34bd843c-konnectivity-ca\") pod \"konnectivity-agent-99khn\" (UID: \"73455279-4b14-4bd5-923a-a23b34bd843c\") " pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:50:38.058548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a518917c-09da-43c3-b61e-f17a704fa9a2-serviceca\") pod \"node-ca-dvh7c\" (UID: \"a518917c-09da-43c3-b61e-f17a704fa9a2\") " pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:38.058548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058522    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w2qx\" (UniqueName: \"kubernetes.io/projected/2217f173-6651-4cfb-86d7-68c3843adf07-kube-api-access-7w2qx\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.058548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058536    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ab2581-3283-415f-b373-d3eedae55bfa-tmp-dir\") pod \"node-resolver-4lbp5\" (UID: \"d0ab2581-3283-415f-b373-d3eedae55bfa\") " pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058552    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-system-cni-dir\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058567    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-cnibin\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058589    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-run-multus-certs\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058604    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058635    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-log-socket\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058666    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-var-lib-openvswitch\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058713    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058738    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-etc-selinux\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44e5c95e-88e3-41b1-88b1-928f8c7f31ae-host-slash\") pod \"iptables-alerter-kt49n\" (UID: \"44e5c95e-88e3-41b1-88b1-928f8c7f31ae\") " pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058784    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-ovn-node-metrics-cert\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058811    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-systemd\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058818    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-etc-selinux\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.058911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44e5c95e-88e3-41b1-88b1-928f8c7f31ae-host-slash\") pod \"iptables-alerter-kt49n\" (UID: \"44e5c95e-88e3-41b1-88b1-928f8c7f31ae\") " pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-run\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbzw7\" (UniqueName: \"kubernetes.io/projected/178ab065-8406-4e3f-8005-b6771517935e-kube-api-access-xbzw7\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058923    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-systemd-units\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058941    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk8pb\" (UniqueName: \"kubernetes.io/projected/0892d712-0444-4ed9-8da9-41d802a56f8b-kube-api-access-lk8pb\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-cni-netd\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058973    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-tuned\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.058997    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-socket-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-run-netns\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/672c6793-5272-43d1-aa5d-ce5bf204f783-kubelet-config\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059083    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/73455279-4b14-4bd5-923a-a23b34bd843c-agent-certs\") pod \"konnectivity-agent-99khn\" (UID: \"73455279-4b14-4bd5-923a-a23b34bd843c\") " pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059113    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-socket-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059132    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-var-lib-kubelet\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059146    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ab2581-3283-415f-b373-d3eedae55bfa-tmp-dir\") pod \"node-resolver-4lbp5\" (UID: \"d0ab2581-3283-415f-b373-d3eedae55bfa\") " pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:38.059434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059158    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-host\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-sys-fs\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-os-release\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-sys-fs\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-var-lib-cni-bin\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059298    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6xch\" (UniqueName: \"kubernetes.io/projected/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-kube-api-access-d6xch\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0ab2581-3283-415f-b373-d3eedae55bfa-hosts-file\") pod \"node-resolver-4lbp5\" (UID: \"d0ab2581-3283-415f-b373-d3eedae55bfa\") " pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-multus-conf-dir\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-run-ovn\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-sys\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0ab2581-3283-415f-b373-d3eedae55bfa-hosts-file\") pod \"node-resolver-4lbp5\" (UID: \"d0ab2581-3283-415f-b373-d3eedae55bfa\") " pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059414    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-multus-cni-dir\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059443    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpx5k\" (UniqueName: \"kubernetes.io/projected/d0ab2581-3283-415f-b373-d3eedae55bfa-kube-api-access-mpx5k\") pod \"node-resolver-4lbp5\" (UID: \"d0ab2581-3283-415f-b373-d3eedae55bfa\") " pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:38.059881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-hostroot\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059475    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-node-log\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059489    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n62qj\" (UniqueName: \"kubernetes.io/projected/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-kube-api-access-n62qj\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059511    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-sysctl-d\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059532    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-run-netns\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059555    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-device-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059575    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/178ab065-8406-4e3f-8005-b6771517935e-multus-daemon-config\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-modprobe-d\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059611    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-device-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059618    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-lib-modules\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059645    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-run-systemd\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059662    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059675    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-multus-socket-dir-parent\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059711    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-var-lib-kubelet\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.060267 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-cni-bin\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059768    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-ovnkube-script-lib\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059795    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-var-lib-cni-multus\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059819    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-etc-kubernetes\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-registration-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059867    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d92cc\" (UniqueName: \"kubernetes.io/projected/44e5c95e-88e3-41b1-88b1-928f8c7f31ae-kube-api-access-d92cc\") pod \"iptables-alerter-kt49n\" (UID: \"44e5c95e-88e3-41b1-88b1-928f8c7f31ae\") " pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-sysconfig\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059898    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-tmp\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059898    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2217f173-6651-4cfb-86d7-68c3843adf07-registration-dir\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059944    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44e5c95e-88e3-41b1-88b1-928f8c7f31ae-iptables-alerter-script\") pod \"iptables-alerter-kt49n\" (UID: \"44e5c95e-88e3-41b1-88b1-928f8c7f31ae\") " pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.059965    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-run-k8s-cni-cncf-io\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.060646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.060346    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44e5c95e-88e3-41b1-88b1-928f8c7f31ae-iptables-alerter-script\") pod \"iptables-alerter-kt49n\" (UID: \"44e5c95e-88e3-41b1-88b1-928f8c7f31ae\") " pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:38.065091 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.064955    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 20 15:50:38.068926 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.068891    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d92cc\" (UniqueName: \"kubernetes.io/projected/44e5c95e-88e3-41b1-88b1-928f8c7f31ae-kube-api-access-d92cc\") pod \"iptables-alerter-kt49n\" (UID: \"44e5c95e-88e3-41b1-88b1-928f8c7f31ae\") " pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:38.069059 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.068925    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w2qx\" (UniqueName: \"kubernetes.io/projected/2217f173-6651-4cfb-86d7-68c3843adf07-kube-api-access-7w2qx\") pod \"aws-ebs-csi-driver-node-gsjkl\" (UID: \"2217f173-6651-4cfb-86d7-68c3843adf07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.069059 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.068925    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpx5k\" (UniqueName: \"kubernetes.io/projected/d0ab2581-3283-415f-b373-d3eedae55bfa-kube-api-access-mpx5k\") pod \"node-resolver-4lbp5\" (UID: \"d0ab2581-3283-415f-b373-d3eedae55bfa\") " pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:38.126703 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.126658    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd6f877677494bfec658ee6d0ff608a.slice/crio-05296f7800a2b387712e9e67bad157b89363bca11874671dcf7d681f697cde86 WatchSource:0}: Error finding container 05296f7800a2b387712e9e67bad157b89363bca11874671dcf7d681f697cde86: Status 404 returned error can't find the container with id 05296f7800a2b387712e9e67bad157b89363bca11874671dcf7d681f697cde86
Jul 20 15:50:38.127463 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.127441    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2bd98933375e305b9c32417ca912b54.slice/crio-d5cbf1453cf023797dc137f6f00449439793eb10d10ae7049a0452da27cab7b0 WatchSource:0}: Error finding container d5cbf1453cf023797dc137f6f00449439793eb10d10ae7049a0452da27cab7b0: Status 404 returned error can't find the container with id d5cbf1453cf023797dc137f6f00449439793eb10d10ae7049a0452da27cab7b0
Jul 20 15:50:38.131925 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.131910    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.160645    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-system-cni-dir\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.160723    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-cnibin\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.160759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-run-multus-certs\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.160793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.160837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-log-socket\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.160870    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-var-lib-openvswitch\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.160905    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-ovn-node-metrics-cert\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.160934    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-systemd\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161010    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-log-socket\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-run\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161103    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbzw7\" (UniqueName: \"kubernetes.io/projected/178ab065-8406-4e3f-8005-b6771517935e-kube-api-access-xbzw7\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.161126    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-systemd-units\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.163546 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.161197    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret podName:672c6793-5272-43d1-aa5d-ce5bf204f783 nodeName:}" failed. No retries permitted until 2026-07-20 15:50:38.661170367 +0000 UTC m=+3.316433637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret") pod "global-pull-secret-syncer-t4wh7" (UID: "672c6793-5272-43d1-aa5d-ce5bf204f783") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-systemd-units\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk8pb\" (UniqueName: \"kubernetes.io/projected/0892d712-0444-4ed9-8da9-41d802a56f8b-kube-api-access-lk8pb\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-cni-netd\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161291    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-tuned\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161321    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-run-netns\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161362    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/672c6793-5272-43d1-aa5d-ce5bf204f783-kubelet-config\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/73455279-4b14-4bd5-923a-a23b34bd843c-agent-certs\") pod \"konnectivity-agent-99khn\" (UID: \"73455279-4b14-4bd5-923a-a23b34bd843c\") " pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-var-lib-kubelet\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161490    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-var-lib-openvswitch\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161565    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-var-lib-kubelet\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-host\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-os-release\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161675    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-var-lib-cni-bin\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161729    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.164342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161764    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-cni-netd\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-system-cni-dir\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162173    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-cnibin\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162261    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-run-multus-certs\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-systemd\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162431    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-run\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.161764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6xch\" (UniqueName: \"kubernetes.io/projected/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-kube-api-access-d6xch\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-multus-conf-dir\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162777    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-run-ovn\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-sys\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162834    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-multus-cni-dir\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162858    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-hostroot\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162886    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-node-log\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162915    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n62qj\" (UniqueName: \"kubernetes.io/projected/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-kube-api-access-n62qj\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-sysctl-d\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.165093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.162977    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-run-netns\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163004    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/178ab065-8406-4e3f-8005-b6771517935e-multus-daemon-config\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-modprobe-d\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163107    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-lib-modules\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163178    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-run-systemd\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163214    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163242    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-multus-socket-dir-parent\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-var-lib-kubelet\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-cni-bin\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163353    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-ovnkube-script-lib\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163381    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-var-lib-cni-multus\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163423    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-etc-kubernetes\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163518    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-sysconfig\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163542    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-tmp\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163572    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-run-k8s-cni-cncf-io\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.165764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163637    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhckn\" (UniqueName: \"kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn\") pod \"network-check-target-hggvm\" (UID: \"b63c3b63-90d2-45eb-ae0d-8491fa36761c\") " pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163823    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-var-lib-cni-bin\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/178ab065-8406-4e3f-8005-b6771517935e-cni-binary-copy\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-env-overrides\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163959    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-etc-openvswitch\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163980    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-host\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.163990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl67g\" (UniqueName: \"kubernetes.io/projected/a518917c-09da-43c3-b61e-f17a704fa9a2-kube-api-access-nl67g\") pod \"node-ca-dvh7c\" (UID: \"a518917c-09da-43c3-b61e-f17a704fa9a2\") " pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164033    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-multus-cni-dir\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-node-log\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164129    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-run-systemd\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164185    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-hostroot\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164247    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-sys\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-run-netns\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.166470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/672c6793-5272-43d1-aa5d-ce5bf204f783-dbus\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164368    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-kubelet\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164390    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-run-netns\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164401    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-run-openvswitch\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-run-openvswitch\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164435    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/178ab065-8406-4e3f-8005-b6771517935e-cni-binary-copy\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-ovnkube-config\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-kubernetes\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164518    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-sysctl-conf\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.164544    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164548    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a518917c-09da-43c3-b61e-f17a704fa9a2-host\") pod \"node-ca-dvh7c\" (UID: \"a518917c-09da-43c3-b61e-f17a704fa9a2\") " pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164584    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-slash\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164603    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-tuned\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.164617    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs podName:0892d712-0444-4ed9-8da9-41d802a56f8b nodeName:}" failed. No retries permitted until 2026-07-20 15:50:38.664600746 +0000 UTC m=+3.319864020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs") pod "network-metrics-daemon-lplvs" (UID: "0892d712-0444-4ed9-8da9-41d802a56f8b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:38.167071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-slash\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164645    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/73455279-4b14-4bd5-923a-a23b34bd843c-konnectivity-ca\") pod \"konnectivity-agent-99khn\" (UID: \"73455279-4b14-4bd5-923a-a23b34bd843c\") " pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a518917c-09da-43c3-b61e-f17a704fa9a2-serviceca\") pod \"node-ca-dvh7c\" (UID: \"a518917c-09da-43c3-b61e-f17a704fa9a2\") " pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164798    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/672c6793-5272-43d1-aa5d-ce5bf204f783-dbus\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164847    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-ovn-node-metrics-cert\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164888    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-os-release\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164921    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-etc-openvswitch\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-multus-socket-dir-parent\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.164977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-env-overrides\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-run-ovn\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-lib-modules\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165192    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-sysctl-d\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165247    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-ovnkube-config\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165386    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-sysconfig\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-var-lib-kubelet\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.167664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165445    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-kubernetes\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165533    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-modprobe-d\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165549    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/73455279-4b14-4bd5-923a-a23b34bd843c-konnectivity-ca\") pod \"konnectivity-agent-99khn\" (UID: \"73455279-4b14-4bd5-923a-a23b34bd843c\") " pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-kubelet\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165636    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-multus-conf-dir\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165733    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-etc-kubernetes\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/672c6793-5272-43d1-aa5d-ce5bf204f783-kubelet-config\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.165960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-etc-sysctl-conf\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.166025    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/178ab065-8406-4e3f-8005-b6771517935e-multus-daemon-config\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.166209    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a518917c-09da-43c3-b61e-f17a704fa9a2-host\") pod \"node-ca-dvh7c\" (UID: \"a518917c-09da-43c3-b61e-f17a704fa9a2\") " pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.166226    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-var-lib-cni-multus\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.166274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-ovnkube-script-lib\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.166295    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/178ab065-8406-4e3f-8005-b6771517935e-host-run-k8s-cni-cncf-io\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.166514    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a518917c-09da-43c3-b61e-f17a704fa9a2-serviceca\") pod \"node-ca-dvh7c\" (UID: \"a518917c-09da-43c3-b61e-f17a704fa9a2\") " pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:38.168160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.166635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-host-cni-bin\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.168791 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.167405    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/73455279-4b14-4bd5-923a-a23b34bd843c-agent-certs\") pod \"konnectivity-agent-99khn\" (UID: \"73455279-4b14-4bd5-923a-a23b34bd843c\") " pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:50:38.168791 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.167451    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-tmp\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.169258 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.169239    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk8pb\" (UniqueName: \"kubernetes.io/projected/0892d712-0444-4ed9-8da9-41d802a56f8b-kube-api-access-lk8pb\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:38.169842 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.169823    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6xch\" (UniqueName: \"kubernetes.io/projected/1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0-kube-api-access-d6xch\") pod \"tuned-w5bhp\" (UID: \"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.170451 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.170436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbzw7\" (UniqueName: \"kubernetes.io/projected/178ab065-8406-4e3f-8005-b6771517935e-kube-api-access-xbzw7\") pod \"multus-t597z\" (UID: \"178ab065-8406-4e3f-8005-b6771517935e\") " pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.174262 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.174240    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:50:38.174354 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.174267    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:50:38.174354 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.174280    2564 projected.go:196] Error preparing data for projected volume kube-api-access-lhckn for pod openshift-network-diagnostics/network-check-target-hggvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:38.174354 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.174342    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn podName:b63c3b63-90d2-45eb-ae0d-8491fa36761c nodeName:}" failed. No retries permitted until 2026-07-20 15:50:38.674318825 +0000 UTC m=+3.329582079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lhckn" (UniqueName: "kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn") pod "network-check-target-hggvm" (UID: "b63c3b63-90d2-45eb-ae0d-8491fa36761c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:38.175561 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.175542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n62qj\" (UniqueName: \"kubernetes.io/projected/6310153e-76c1-4dc3-bfc2-e36cc217b2e1-kube-api-access-n62qj\") pod \"ovnkube-node-kxp2n\" (UID: \"6310153e-76c1-4dc3-bfc2-e36cc217b2e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.176230 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.176213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl67g\" (UniqueName: \"kubernetes.io/projected/a518917c-09da-43c3-b61e-f17a704fa9a2-kube-api-access-nl67g\") pod \"node-ca-dvh7c\" (UID: \"a518917c-09da-43c3-b61e-f17a704fa9a2\") " pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:38.277149 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.277113    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl"
Jul 20 15:50:38.283515 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.283498    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4lbp5"
Jul 20 15:50:38.287413 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.287385    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2217f173_6651_4cfb_86d7_68c3843adf07.slice/crio-3389522760e383893e436080696fe5a8458f3e6f8692d3b80fb8f375503e1575 WatchSource:0}: Error finding container 3389522760e383893e436080696fe5a8458f3e6f8692d3b80fb8f375503e1575: Status 404 returned error can't find the container with id 3389522760e383893e436080696fe5a8458f3e6f8692d3b80fb8f375503e1575
Jul 20 15:50:38.291399 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.291378    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ab2581_3283_415f_b373_d3eedae55bfa.slice/crio-4da1ed34ff2adc89a80cbf88aa6bd32ba52133020660d99c8aa9e37477f865af WatchSource:0}: Error finding container 4da1ed34ff2adc89a80cbf88aa6bd32ba52133020660d99c8aa9e37477f865af: Status 404 returned error can't find the container with id 4da1ed34ff2adc89a80cbf88aa6bd32ba52133020660d99c8aa9e37477f865af
Jul 20 15:50:38.303822 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.303799    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kt49n"
Jul 20 15:50:38.312950 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.312927    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e5c95e_88e3_41b1_88b1_928f8c7f31ae.slice/crio-d878fa1a300119d4c6f2940c334c1f6255e00c11d864baf05fc0134018ba91c1 WatchSource:0}: Error finding container d878fa1a300119d4c6f2940c334c1f6255e00c11d864baf05fc0134018ba91c1: Status 404 returned error can't find the container with id d878fa1a300119d4c6f2940c334c1f6255e00c11d864baf05fc0134018ba91c1
Jul 20 15:50:38.322553 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.322535    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t597z"
Jul 20 15:50:38.327887 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.327613    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:50:38.333574 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.333554    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod178ab065_8406_4e3f_8005_b6771517935e.slice/crio-1fdb9509b290cbc0b33f36e8fd0c2ecea1526bdea609145aa98999733ba4abfe WatchSource:0}: Error finding container 1fdb9509b290cbc0b33f36e8fd0c2ecea1526bdea609145aa98999733ba4abfe: Status 404 returned error can't find the container with id 1fdb9509b290cbc0b33f36e8fd0c2ecea1526bdea609145aa98999733ba4abfe
Jul 20 15:50:38.336164 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.336143    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6310153e_76c1_4dc3_bfc2_e36cc217b2e1.slice/crio-a06002c684751bd9d0100c2c6e58a794c4e6b5db78e72ac01b1be99d0c4eb494 WatchSource:0}: Error finding container a06002c684751bd9d0100c2c6e58a794c4e6b5db78e72ac01b1be99d0c4eb494: Status 404 returned error can't find the container with id a06002c684751bd9d0100c2c6e58a794c4e6b5db78e72ac01b1be99d0c4eb494
Jul 20 15:50:38.353429 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.353405    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:50:38.362479 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.362460    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73455279_4b14_4bd5_923a_a23b34bd843c.slice/crio-2bb9c03006af6e16957a50ff839d70b364dae5b0fec10de59cb4fe52952c3118 WatchSource:0}: Error finding container 2bb9c03006af6e16957a50ff839d70b364dae5b0fec10de59cb4fe52952c3118: Status 404 returned error can't find the container with id 2bb9c03006af6e16957a50ff839d70b364dae5b0fec10de59cb4fe52952c3118
Jul 20 15:50:38.366580 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.366560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:38.366636 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.366592    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:38.366636 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.366623    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b69s\" (UniqueName: \"kubernetes.io/projected/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-kube-api-access-5b69s\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:38.366753 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.366736    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-binary-copy\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:38.367110 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.367092    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:38.367581 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.367562    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:38.367665 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.367655    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w5bhp"
Jul 20 15:50:38.367806 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.367784    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-cni-binary-copy\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:38.369062 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.369045    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b69s\" (UniqueName: \"kubernetes.io/projected/a46171f5-26e9-4fda-9f5d-6e1ef9a0995e-kube-api-access-5b69s\") pod \"multus-additional-cni-plugins-8p6lq\" (UID: \"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e\") " pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:38.376153 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.376135    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1634ddcf_81b3_4ed4_b7e2_9b4cbd22c0a0.slice/crio-00834924b6eedc2abe3eb96b8dccb6776ac8a4a77dd4a4f2738875b75250247a WatchSource:0}: Error finding container 00834924b6eedc2abe3eb96b8dccb6776ac8a4a77dd4a4f2738875b75250247a: Status 404 returned error can't find the container with id 00834924b6eedc2abe3eb96b8dccb6776ac8a4a77dd4a4f2738875b75250247a
Jul 20 15:50:38.379033 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.379018    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dvh7c"
Jul 20 15:50:38.386989 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.386959    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda518917c_09da_43c3_b61e_f17a704fa9a2.slice/crio-0a29779a3af386bca15a3da2f36d1419c7af83d1694ed28ccf1767ea7f81ab62 WatchSource:0}: Error finding container 0a29779a3af386bca15a3da2f36d1419c7af83d1694ed28ccf1767ea7f81ab62: Status 404 returned error can't find the container with id 0a29779a3af386bca15a3da2f36d1419c7af83d1694ed28ccf1767ea7f81ab62
Jul 20 15:50:38.560855 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.560698    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8p6lq"
Jul 20 15:50:38.570945 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:50:38.570909    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda46171f5_26e9_4fda_9f5d_6e1ef9a0995e.slice/crio-1927fedd6f5956d705d5774d9265a3967f2348e5e55ed8a515e078caa441a761 WatchSource:0}: Error finding container 1927fedd6f5956d705d5774d9265a3967f2348e5e55ed8a515e078caa441a761: Status 404 returned error can't find the container with id 1927fedd6f5956d705d5774d9265a3967f2348e5e55ed8a515e078caa441a761
Jul 20 15:50:38.670171 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.669979    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:38.670171 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.670064    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:38.670171 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.670168    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:38.670449 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.670184    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:38.670449 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.670245    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs podName:0892d712-0444-4ed9-8da9-41d802a56f8b nodeName:}" failed. No retries permitted until 2026-07-20 15:50:39.67022582 +0000 UTC m=+4.325489079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs") pod "network-metrics-daemon-lplvs" (UID: "0892d712-0444-4ed9-8da9-41d802a56f8b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:38.670449 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.670267    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret podName:672c6793-5272-43d1-aa5d-ce5bf204f783 nodeName:}" failed. No retries permitted until 2026-07-20 15:50:39.670258397 +0000 UTC m=+4.325521655 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret") pod "global-pull-secret-syncer-t4wh7" (UID: "672c6793-5272-43d1-aa5d-ce5bf204f783") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:38.771419 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.771381    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhckn\" (UniqueName: \"kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn\") pod \"network-check-target-hggvm\" (UID: \"b63c3b63-90d2-45eb-ae0d-8491fa36761c\") " pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:38.771607 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.771572    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:50:38.771607 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.771596    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:50:38.771732 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.771609    2564 projected.go:196] Error preparing data for projected volume kube-api-access-lhckn for pod openshift-network-diagnostics/network-check-target-hggvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:38.771732 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:38.771670    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn podName:b63c3b63-90d2-45eb-ae0d-8491fa36761c nodeName:}" failed. No retries permitted until 2026-07-20 15:50:39.771650985 +0000 UTC m=+4.426914256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lhckn" (UniqueName: "kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn") pod "network-check-target-hggvm" (UID: "b63c3b63-90d2-45eb-ae0d-8491fa36761c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:38.876161 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.876043    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t597z" event={"ID":"178ab065-8406-4e3f-8005-b6771517935e","Type":"ContainerStarted","Data":"1fdb9509b290cbc0b33f36e8fd0c2ecea1526bdea609145aa98999733ba4abfe"}
Jul 20 15:50:38.895069 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.894922    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kt49n" event={"ID":"44e5c95e-88e3-41b1-88b1-928f8c7f31ae","Type":"ContainerStarted","Data":"d878fa1a300119d4c6f2940c334c1f6255e00c11d864baf05fc0134018ba91c1"}
Jul 20 15:50:38.898249 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.898217    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" event={"ID":"6cd6f877677494bfec658ee6d0ff608a","Type":"ContainerStarted","Data":"05296f7800a2b387712e9e67bad157b89363bca11874671dcf7d681f697cde86"}
Jul 20 15:50:38.911760 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.911723    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8p6lq" event={"ID":"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e","Type":"ContainerStarted","Data":"1927fedd6f5956d705d5774d9265a3967f2348e5e55ed8a515e078caa441a761"}
Jul 20 15:50:38.917617 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.917490    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dvh7c" event={"ID":"a518917c-09da-43c3-b61e-f17a704fa9a2","Type":"ContainerStarted","Data":"0a29779a3af386bca15a3da2f36d1419c7af83d1694ed28ccf1767ea7f81ab62"}
Jul 20 15:50:38.926762 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.926725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-99khn" event={"ID":"73455279-4b14-4bd5-923a-a23b34bd843c","Type":"ContainerStarted","Data":"2bb9c03006af6e16957a50ff839d70b364dae5b0fec10de59cb4fe52952c3118"}
Jul 20 15:50:38.932588 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.932550    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" event={"ID":"6310153e-76c1-4dc3-bfc2-e36cc217b2e1","Type":"ContainerStarted","Data":"a06002c684751bd9d0100c2c6e58a794c4e6b5db78e72ac01b1be99d0c4eb494"}
Jul 20 15:50:38.941130 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.941071    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 15:45:37 +0000 UTC" deadline="2028-04-15 22:02:38.21273166 +0000 UTC"
Jul 20 15:50:38.941130 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.941129    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15246h11m59.271607002s"
Jul 20 15:50:38.943557 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.943523    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4lbp5" event={"ID":"d0ab2581-3283-415f-b373-d3eedae55bfa","Type":"ContainerStarted","Data":"4da1ed34ff2adc89a80cbf88aa6bd32ba52133020660d99c8aa9e37477f865af"}
Jul 20 15:50:38.949504 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.949475    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl" event={"ID":"2217f173-6651-4cfb-86d7-68c3843adf07","Type":"ContainerStarted","Data":"3389522760e383893e436080696fe5a8458f3e6f8692d3b80fb8f375503e1575"}
Jul 20 15:50:38.955939 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.955894    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" event={"ID":"c2bd98933375e305b9c32417ca912b54","Type":"ContainerStarted","Data":"d5cbf1453cf023797dc137f6f00449439793eb10d10ae7049a0452da27cab7b0"}
Jul 20 15:50:38.960325 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:38.960286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w5bhp" event={"ID":"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0","Type":"ContainerStarted","Data":"00834924b6eedc2abe3eb96b8dccb6776ac8a4a77dd4a4f2738875b75250247a"}
Jul 20 15:50:39.643718 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:39.643671    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:50:39.679967 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:39.679923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:39.680200 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:39.680018    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:39.680200 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.680146    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:39.680321 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.680206    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret podName:672c6793-5272-43d1-aa5d-ce5bf204f783 nodeName:}" failed. No retries permitted until 2026-07-20 15:50:41.680188739 +0000 UTC m=+6.335452000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret") pod "global-pull-secret-syncer-t4wh7" (UID: "672c6793-5272-43d1-aa5d-ce5bf204f783") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:39.680578 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.680562    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:39.680655 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.680611    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs podName:0892d712-0444-4ed9-8da9-41d802a56f8b nodeName:}" failed. No retries permitted until 2026-07-20 15:50:41.680596098 +0000 UTC m=+6.335859408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs") pod "network-metrics-daemon-lplvs" (UID: "0892d712-0444-4ed9-8da9-41d802a56f8b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:39.780904 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:39.780768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhckn\" (UniqueName: \"kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn\") pod \"network-check-target-hggvm\" (UID: \"b63c3b63-90d2-45eb-ae0d-8491fa36761c\") " pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:39.781067 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.780954    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:50:39.781067 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.780981    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:50:39.781067 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.780994    2564 projected.go:196] Error preparing data for projected volume kube-api-access-lhckn for pod openshift-network-diagnostics/network-check-target-hggvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:39.781067 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.781055    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn podName:b63c3b63-90d2-45eb-ae0d-8491fa36761c nodeName:}" failed. No retries permitted until 2026-07-20 15:50:41.781035903 +0000 UTC m=+6.436299174 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lhckn" (UniqueName: "kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn") pod "network-check-target-hggvm" (UID: "b63c3b63-90d2-45eb-ae0d-8491fa36761c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:39.864114 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:39.863186    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:39.864114 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.863322    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:39.864114 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:39.863746    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:39.864114 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.863855    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:39.864114 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:39.863968    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:39.864114 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:39.864051    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:39.942021 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:39.941929    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 15:45:37 +0000 UTC" deadline="2028-04-06 22:19:00.491624082 +0000 UTC"
Jul 20 15:50:39.942021 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:39.941973    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15030h28m20.549654855s"
Jul 20 15:50:41.696678 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:41.695851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:41.696678 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:41.695934    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:41.696678 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.696064    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:41.696678 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.696124    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs podName:0892d712-0444-4ed9-8da9-41d802a56f8b nodeName:}" failed. No retries permitted until 2026-07-20 15:50:45.69610536 +0000 UTC m=+10.351368654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs") pod "network-metrics-daemon-lplvs" (UID: "0892d712-0444-4ed9-8da9-41d802a56f8b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:41.696678 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.696538    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:41.696678 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.696590    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret podName:672c6793-5272-43d1-aa5d-ce5bf204f783 nodeName:}" failed. No retries permitted until 2026-07-20 15:50:45.696574221 +0000 UTC m=+10.351837500 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret") pod "global-pull-secret-syncer-t4wh7" (UID: "672c6793-5272-43d1-aa5d-ce5bf204f783") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:41.796971 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:41.796932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhckn\" (UniqueName: \"kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn\") pod \"network-check-target-hggvm\" (UID: \"b63c3b63-90d2-45eb-ae0d-8491fa36761c\") " pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:41.797160 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.797120    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:50:41.797160 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.797145    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:50:41.797160 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.797157    2564 projected.go:196] Error preparing data for projected volume kube-api-access-lhckn for pod openshift-network-diagnostics/network-check-target-hggvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:41.797305 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.797218    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn podName:b63c3b63-90d2-45eb-ae0d-8491fa36761c nodeName:}" failed. No retries permitted until 2026-07-20 15:50:45.797197717 +0000 UTC m=+10.452460993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lhckn" (UniqueName: "kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn") pod "network-check-target-hggvm" (UID: "b63c3b63-90d2-45eb-ae0d-8491fa36761c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:41.863738 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:41.863703    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:41.863906 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.863838    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:41.864368 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:41.864006    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:41.864368 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:41.864101    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:41.864368 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.864209    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:41.864368 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:41.864300    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:43.863633 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:43.863599    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:43.864064 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:43.863599    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:43.864064 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:43.863750    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:43.864064 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:43.863605    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:43.864064 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:43.863816    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:43.864064 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:43.863878    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:45.731259 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:45.731203    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:45.731893 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:45.731288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:45.731893 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.731431    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:45.731893 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.731492    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs podName:0892d712-0444-4ed9-8da9-41d802a56f8b nodeName:}" failed. No retries permitted until 2026-07-20 15:50:53.731472236 +0000 UTC m=+18.386735509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs") pod "network-metrics-daemon-lplvs" (UID: "0892d712-0444-4ed9-8da9-41d802a56f8b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:45.732058 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.731928    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:45.732058 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.731981    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret podName:672c6793-5272-43d1-aa5d-ce5bf204f783 nodeName:}" failed. No retries permitted until 2026-07-20 15:50:53.731963867 +0000 UTC m=+18.387227156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret") pod "global-pull-secret-syncer-t4wh7" (UID: "672c6793-5272-43d1-aa5d-ce5bf204f783") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:45.832299 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:45.832205    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhckn\" (UniqueName: \"kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn\") pod \"network-check-target-hggvm\" (UID: \"b63c3b63-90d2-45eb-ae0d-8491fa36761c\") " pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:45.832466 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.832389    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:50:45.832466 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.832413    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:50:45.832466 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.832425    2564 projected.go:196] Error preparing data for projected volume kube-api-access-lhckn for pod openshift-network-diagnostics/network-check-target-hggvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:45.832609 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.832493    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn podName:b63c3b63-90d2-45eb-ae0d-8491fa36761c nodeName:}" failed. No retries permitted until 2026-07-20 15:50:53.83247445 +0000 UTC m=+18.487737729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lhckn" (UniqueName: "kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn") pod "network-check-target-hggvm" (UID: "b63c3b63-90d2-45eb-ae0d-8491fa36761c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:45.864350 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:45.864318    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:45.864525 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.864422    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:45.864938 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:45.864626    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:45.864938 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:45.864728    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:45.864938 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.864828    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:45.864938 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:45.864908    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:47.863625 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:47.863580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:47.864093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:47.863580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:47.864093 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:47.863720    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:47.864093 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:47.863821    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:47.864093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:47.863598    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:47.864093 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:47.863914    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:49.863673 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:49.863642    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:49.863673 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:49.863661    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:49.864167 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:49.863795    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:49.864167 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:49.863859    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:49.864167 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:49.863895    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:49.864167 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:49.863968    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:51.863519 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:51.863482    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:51.864017 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:51.863482    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:51.864017 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:51.863601    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:51.864017 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:51.863701    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:51.864017 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:51.863482    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:51.864017 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:51.863795    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:53.794398 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:53.794338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:53.794911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:53.794442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:53.794911 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.794518    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:53.794911 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.794533    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:53.794911 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.794586    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs podName:0892d712-0444-4ed9-8da9-41d802a56f8b nodeName:}" failed. No retries permitted until 2026-07-20 15:51:09.794569575 +0000 UTC m=+34.449832845 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs") pod "network-metrics-daemon-lplvs" (UID: "0892d712-0444-4ed9-8da9-41d802a56f8b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:50:53.794911 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.794602    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret podName:672c6793-5272-43d1-aa5d-ce5bf204f783 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:09.794595391 +0000 UTC m=+34.449858692 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret") pod "global-pull-secret-syncer-t4wh7" (UID: "672c6793-5272-43d1-aa5d-ce5bf204f783") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:50:53.863260 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:53.863224    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:53.863412 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:53.863224    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:53.863412 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:53.863224    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:53.863412 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.863368    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:53.863577 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.863458    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:53.863638 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.863576    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:53.894963 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:53.894929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhckn\" (UniqueName: \"kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn\") pod \"network-check-target-hggvm\" (UID: \"b63c3b63-90d2-45eb-ae0d-8491fa36761c\") " pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:53.895133 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.895107    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:50:53.895133 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.895130    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:50:53.895224 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.895142    2564 projected.go:196] Error preparing data for projected volume kube-api-access-lhckn for pod openshift-network-diagnostics/network-check-target-hggvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:53.895224 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:53.895216    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn podName:b63c3b63-90d2-45eb-ae0d-8491fa36761c nodeName:}" failed. No retries permitted until 2026-07-20 15:51:09.895196128 +0000 UTC m=+34.550459401 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lhckn" (UniqueName: "kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn") pod "network-check-target-hggvm" (UID: "b63c3b63-90d2-45eb-ae0d-8491fa36761c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:50:55.864208 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:55.864175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:55.864610 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:55.864232    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:55.864610 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:55.864551    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:55.864750 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:55.864710    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:55.864805 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:55.864750    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:55.864924 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:55.864895    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:57.863469 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:57.863437    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:57.863859 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:57.863438    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:57.863859 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:57.863548    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:57.863859 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:57.863561    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:57.863859 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:57.863651    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:50:57.863859 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:57.863735    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:59.018179 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.018033    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 15:50:59.018587 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.018540    2564 generic.go:356] "Generic (PLEG): container finished" podID="6310153e-76c1-4dc3-bfc2-e36cc217b2e1" containerID="4eca4e8180ccf9e77c46ea8eb02229c3b04c148ac5eb1334b29ee948f84b097a" exitCode=1
Jul 20 15:50:59.018646 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.018586    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" event={"ID":"6310153e-76c1-4dc3-bfc2-e36cc217b2e1","Type":"ContainerStarted","Data":"101d3ec5738db65fa0704b2e3293f63d1a27dbae95123bb53f3027c7c98623a6"}
Jul 20 15:50:59.018737 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.018718    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" event={"ID":"6310153e-76c1-4dc3-bfc2-e36cc217b2e1","Type":"ContainerDied","Data":"4eca4e8180ccf9e77c46ea8eb02229c3b04c148ac5eb1334b29ee948f84b097a"}
Jul 20 15:50:59.018797 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.018781    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" event={"ID":"6310153e-76c1-4dc3-bfc2-e36cc217b2e1","Type":"ContainerStarted","Data":"bd6ce92d1911d5db89ae438915df27dd3b6b48728308b81203fdc815a14903d8"}
Jul 20 15:50:59.020145 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.020124    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" event={"ID":"c2bd98933375e305b9c32417ca912b54","Type":"ContainerStarted","Data":"b1c4fce7c61bad35b0adefc00d755c4df4b2d651d76303382283dc01a5174643"}
Jul 20 15:50:59.021912 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.021874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w5bhp" event={"ID":"1634ddcf-81b3-4ed4-b7e2-9b4cbd22c0a0","Type":"ContainerStarted","Data":"0b356ac86d9363d78dc8707e9db857e0db53666f89a759f1864ef8ba7f86cd17"}
Jul 20 15:50:59.023422 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.023403    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t597z" event={"ID":"178ab065-8406-4e3f-8005-b6771517935e","Type":"ContainerStarted","Data":"a127537d8375e3055af6c10bd96aa3a8d6fe896033370475ad52bd44978846b4"}
Jul 20 15:50:59.033928 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.033886    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" podStartSLOduration=23.033874145 podStartE2EDuration="23.033874145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:50:59.033749426 +0000 UTC m=+23.689012703" watchObservedRunningTime="2026-07-20 15:50:59.033874145 +0000 UTC m=+23.689137420"
Jul 20 15:50:59.050270 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.050156    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w5bhp" podStartSLOduration=3.323789556 podStartE2EDuration="23.050143789s" totalImagesPullingTime="19.726354233s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:50:59.049587572 +0000 UTC m=+23.704850847" watchObservedRunningTime="2026-07-20 15:50:59.050143789 +0000 UTC m=+23.705407066"
Jul 20 15:50:59.068096 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.068047    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-t597z" podStartSLOduration=3.285868604 podStartE2EDuration="23.068031817s" totalImagesPullingTime="19.782163213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:50:59.067211564 +0000 UTC m=+23.722475042" watchObservedRunningTime="2026-07-20 15:50:59.068031817 +0000 UTC m=+23.723295133"
Jul 20 15:50:59.863846 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.863809    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:50:59.863846 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.863830    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:50:59.864097 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:50:59.863956    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:50:59.864097 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:59.863952    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:50:59.864097 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:59.864087    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:50:59.864239 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:50:59.864195    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:51:00.027173 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.027137    2564 generic.go:356] "Generic (PLEG): container finished" podID="a46171f5-26e9-4fda-9f5d-6e1ef9a0995e" containerID="f882117f9cb90269ecbf2f33f9f62901c53e953890f1958e7fabb53d22364bf9" exitCode=0
Jul 20 15:51:00.027947 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.027219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8p6lq" event={"ID":"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e","Type":"ContainerDied","Data":"f882117f9cb90269ecbf2f33f9f62901c53e953890f1958e7fabb53d22364bf9"}
Jul 20 15:51:00.028861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.028759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dvh7c" event={"ID":"a518917c-09da-43c3-b61e-f17a704fa9a2","Type":"ContainerStarted","Data":"d014d11ba33d9cbb39f623da02a01e06341e9fd44d9b9cb410025b581bf3c42a"}
Jul 20 15:51:00.030411 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.030387    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-99khn" event={"ID":"73455279-4b14-4bd5-923a-a23b34bd843c","Type":"ContainerStarted","Data":"18e185b5f8d27cde024904c37c63f47994196ec7743bab520489e4a93c59b12f"}
Jul 20 15:51:00.033710 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.033679    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 15:51:00.034062 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.034045    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" event={"ID":"6310153e-76c1-4dc3-bfc2-e36cc217b2e1","Type":"ContainerStarted","Data":"d50fd43d175fc00cb111dbc48918e771382fb6ae5501f6933bcf878aa666214a"}
Jul 20 15:51:00.034135 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.034117    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" event={"ID":"6310153e-76c1-4dc3-bfc2-e36cc217b2e1","Type":"ContainerStarted","Data":"e29f30278f79e3d81d6615c4f74bc4350c2843afc9c95c5bdbca39c8e3c4288e"}
Jul 20 15:51:00.034194 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.034169    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" event={"ID":"6310153e-76c1-4dc3-bfc2-e36cc217b2e1","Type":"ContainerStarted","Data":"faf3ff634ca15b1f457abcd897a0b52339e505a223654af3a81ba47613af9e07"}
Jul 20 15:51:00.035362 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.035341    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4lbp5" event={"ID":"d0ab2581-3283-415f-b373-d3eedae55bfa","Type":"ContainerStarted","Data":"516ca0208a595d3bc4ecec6e72da6a93b7ec203ffc18138e94eaee0c58893c2c"}
Jul 20 15:51:00.036611 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.036591    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl" event={"ID":"2217f173-6651-4cfb-86d7-68c3843adf07","Type":"ContainerStarted","Data":"c2bd3157602120747dc9f489aa254059c8170c22ff8b8e455edd3a917f7f81a7"}
Jul 20 15:51:00.037824 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.037806    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kt49n" event={"ID":"44e5c95e-88e3-41b1-88b1-928f8c7f31ae","Type":"ContainerStarted","Data":"fcb973df6dff1dc32a99152388697fcea5527ca2e90ec0efb5c237fc843206ed"}
Jul 20 15:51:00.039219 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.039196    2564 generic.go:356] "Generic (PLEG): container finished" podID="6cd6f877677494bfec658ee6d0ff608a" containerID="3d6b5491e6b073cd9f52b870de87ca47c38a379249af92e595499c593b101bdb" exitCode=0
Jul 20 15:51:00.039321 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.039300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" event={"ID":"6cd6f877677494bfec658ee6d0ff608a","Type":"ContainerDied","Data":"3d6b5491e6b073cd9f52b870de87ca47c38a379249af92e595499c593b101bdb"}
Jul 20 15:51:00.075073 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.075021    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-99khn" podStartSLOduration=4.386913607 podStartE2EDuration="24.075006672s" totalImagesPullingTime="19.688093065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:00.074803483 +0000 UTC m=+24.730066758" watchObservedRunningTime="2026-07-20 15:51:00.075006672 +0000 UTC m=+24.730269947"
Jul 20 15:51:00.104356 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.104299    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kt49n" podStartSLOduration=5.3664375 podStartE2EDuration="25.104285142s" totalImagesPullingTime="19.737847642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:00.104281456 +0000 UTC m=+24.759544734" watchObservedRunningTime="2026-07-20 15:51:00.104285142 +0000 UTC m=+24.759548447"
Jul 20 15:51:00.104505 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.104465    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4lbp5" podStartSLOduration=5.3452782899999995 podStartE2EDuration="25.104460981s" totalImagesPullingTime="19.759182691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:00.090232936 +0000 UTC m=+24.745496212" watchObservedRunningTime="2026-07-20 15:51:00.104460981 +0000 UTC m=+24.759724256"
Jul 20 15:51:00.199658 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.199573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:00.275955 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.275918    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:00.327995 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.327787    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:00.620473 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.620437    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 20 15:51:00.850026 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.849914    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T15:51:00.620465048Z","UUID":"bb10d202-a234-4d99-aa53-856ea7979594","Handler":null,"Name":"","Endpoint":""}
Jul 20 15:51:00.851416 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.851393    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 20 15:51:00.851536 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:00.851432    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 20 15:51:01.043097 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:01.043055    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" event={"ID":"6cd6f877677494bfec658ee6d0ff608a","Type":"ContainerStarted","Data":"5a86300b5f6e52bb134d1e6a67be62f10e8f8a0653095b647f1a6c3d93ad9dc6"}
Jul 20 15:51:01.045204 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:01.045138    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl" event={"ID":"2217f173-6651-4cfb-86d7-68c3843adf07","Type":"ContainerStarted","Data":"d6721582d304ee1d7d89981e64e88d96d9daac865c2569f36af85ca4a73d407d"}
Jul 20 15:51:01.060577 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:01.060520    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dvh7c" podStartSLOduration=5.396369687 podStartE2EDuration="25.060501399s" totalImagesPullingTime="19.664131712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:00.126761762 +0000 UTC m=+24.782025039" watchObservedRunningTime="2026-07-20 15:51:01.060501399 +0000 UTC m=+25.715764677"
Jul 20 15:51:01.060767 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:01.060615    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" podStartSLOduration=25.060609322 podStartE2EDuration="25.060609322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:01.059392019 +0000 UTC m=+25.714655296" watchObservedRunningTime="2026-07-20 15:51:01.060609322 +0000 UTC m=+25.715872597"
Jul 20 15:51:01.863603 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:01.863524    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:01.863818 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:01.863536    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:01.863818 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:01.863652    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:51:01.863818 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:01.863660    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:01.863818 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:01.863795    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:51:01.864007 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:01.863883    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:51:02.051084 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:02.051022    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 15:51:02.051876 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:02.051432    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" event={"ID":"6310153e-76c1-4dc3-bfc2-e36cc217b2e1","Type":"ContainerStarted","Data":"ca250c5d8aa70498db86e2d3f6e41eea4cfc2a43a9d47ce24e29008d0a1ba1d8"}
Jul 20 15:51:02.053745 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:02.053713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl" event={"ID":"2217f173-6651-4cfb-86d7-68c3843adf07","Type":"ContainerStarted","Data":"2c25cb8cab2aba9ca4aa40340ec60bce5eadecacdbbf34bceb3eb3a33a3f7ece"}
Jul 20 15:51:03.272281 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:03.272247    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:03.347356 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:03.347307    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:03.400354 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:03.400321    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:03.498293 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:03.498262    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:03.553492 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:03.553412    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:03.863431 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:03.863339    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:03.863593 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:03.863350    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:03.863593 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:03.863473    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:51:03.863593 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:03.863560    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:51:03.863794 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:03.863629    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:03.863794 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:03.863735    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:51:03.880166 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:03.880091    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:03.972162 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:03.972123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:04.169306 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:04.169223    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:04.218718 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:04.218669    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:04.577244 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:04.577211    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:04.632384 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:04.632351    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:04.951407 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:04.951315    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:51:04.952100 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:04.952082    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:51:04.972144 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:04.972092    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsjkl" podStartSLOduration=7.720499257 podStartE2EDuration="29.972076092s" totalImagesPullingTime="22.251576835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:02.075954793 +0000 UTC m=+26.731218082" watchObservedRunningTime="2026-07-20 15:51:04.972076092 +0000 UTC m=+29.627339430"
Jul 20 15:51:05.061018 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:05.060985    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:51:05.061509 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:05.061496    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-99khn"
Jul 20 15:51:05.136136 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:05.136108    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:05.374983 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:05.374575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:05.425330 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:05.425304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:05.764474 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:05.764393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:05.818106 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:05.818078    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:05.863706 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:05.863658    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:05.863886 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:05.863766    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:51:05.863886 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:05.863781    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:05.863886 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:05.863800    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:05.863886 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:05.863875    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:51:05.864097 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:05.863948    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:51:06.064859 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:06.064824    2564 generic.go:356] "Generic (PLEG): container finished" podID="a46171f5-26e9-4fda-9f5d-6e1ef9a0995e" containerID="9cecea9a8123829138d186674ba77b0e099ab62e946eb494a55570f786767140" exitCode=0
Jul 20 15:51:06.065031 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:06.064907    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8p6lq" event={"ID":"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e","Type":"ContainerDied","Data":"9cecea9a8123829138d186674ba77b0e099ab62e946eb494a55570f786767140"}
Jul 20 15:51:06.070229 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:06.070214    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 15:51:06.070661 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:06.070641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" event={"ID":"6310153e-76c1-4dc3-bfc2-e36cc217b2e1","Type":"ContainerStarted","Data":"cb6c43bd3c6f8d0b6c2f550beb627ae8994d061ebb39c0f9c0ec05b69a78d9c3"}
Jul 20 15:51:06.071031 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:06.071016    2564 scope.go:117] "RemoveContainer" containerID="4eca4e8180ccf9e77c46ea8eb02229c3b04c148ac5eb1334b29ee948f84b097a"
Jul 20 15:51:07.075820 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.075796    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 15:51:07.076237 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.076132    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" event={"ID":"6310153e-76c1-4dc3-bfc2-e36cc217b2e1","Type":"ContainerStarted","Data":"c1826d3ea61cee6be89075bc7e5132afd7586bbe950c344721857e13b2987354"}
Jul 20 15:51:07.076237 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.076203    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 20 15:51:07.076540 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.076504    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:51:07.076623 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.076561    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:51:07.093873 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.093838    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:51:07.094405 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.094381    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:51:07.111843 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.111788    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n" podStartSLOduration=10.823898067 podStartE2EDuration="31.111767241s" totalImagesPullingTime="20.287869174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:07.111106739 +0000 UTC m=+31.766370009" watchObservedRunningTime="2026-07-20 15:51:07.111767241 +0000 UTC m=+31.767030530"
Jul 20 15:51:07.207061 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.207028    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t4wh7"]
Jul 20 15:51:07.207236 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.207200    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:07.207338 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:07.207317    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:51:07.213853 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.213822    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hggvm"]
Jul 20 15:51:07.214006 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.213947    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:07.214071 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:07.214049    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:51:07.214473 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.214445    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lplvs"]
Jul 20 15:51:07.214574 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.214552    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:07.214660 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:07.214632    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:51:07.290058 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:07.290028    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:51:08.080343 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:08.080265    2564 generic.go:356] "Generic (PLEG): container finished" podID="a46171f5-26e9-4fda-9f5d-6e1ef9a0995e" containerID="2757ae4df6b0a81a90c45fd848c63d75e5e1d5234a088a3171bb72f2bcf430da" exitCode=0
Jul 20 15:51:08.080721 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:08.080366    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8p6lq" event={"ID":"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e","Type":"ContainerDied","Data":"2757ae4df6b0a81a90c45fd848c63d75e5e1d5234a088a3171bb72f2bcf430da"}
Jul 20 15:51:08.863588 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:08.863552    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:08.863588 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:08.863578    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:08.863806 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:08.863553    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:08.863806 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:08.863664    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:51:08.863806 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:08.863749    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:51:08.863904 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:08.863821    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:51:09.803302 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:09.803269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:09.803664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:09.803321    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:09.803664 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:09.803417    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:51:09.803664 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:09.803421    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:51:09.803664 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:09.803466    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret podName:672c6793-5272-43d1-aa5d-ce5bf204f783 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:41.803454189 +0000 UTC m=+66.458717444 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret") pod "global-pull-secret-syncer-t4wh7" (UID: "672c6793-5272-43d1-aa5d-ce5bf204f783") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:51:09.803664 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:09.803481    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs podName:0892d712-0444-4ed9-8da9-41d802a56f8b nodeName:}" failed. No retries permitted until 2026-07-20 15:51:41.803474087 +0000 UTC m=+66.458737341 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs") pod "network-metrics-daemon-lplvs" (UID: "0892d712-0444-4ed9-8da9-41d802a56f8b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:51:09.904200 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:09.904124    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhckn\" (UniqueName: \"kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn\") pod \"network-check-target-hggvm\" (UID: \"b63c3b63-90d2-45eb-ae0d-8491fa36761c\") " pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:09.904337 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:09.904289    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:51:09.904337 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:09.904317    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:51:09.904337 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:09.904328    2564 projected.go:196] Error preparing data for projected volume kube-api-access-lhckn for pod openshift-network-diagnostics/network-check-target-hggvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:51:09.904435 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:09.904379    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn podName:b63c3b63-90d2-45eb-ae0d-8491fa36761c nodeName:}" failed. No retries permitted until 2026-07-20 15:51:41.904363372 +0000 UTC m=+66.559626646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lhckn" (UniqueName: "kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn") pod "network-check-target-hggvm" (UID: "b63c3b63-90d2-45eb-ae0d-8491fa36761c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:51:10.087262 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:10.087221    2564 generic.go:356] "Generic (PLEG): container finished" podID="a46171f5-26e9-4fda-9f5d-6e1ef9a0995e" containerID="31dd1f44831cfedb6d1a2a95cdfef25fef46b87f262529c5d8ffc0472c1b6c7a" exitCode=0
Jul 20 15:51:10.087433 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:10.087286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8p6lq" event={"ID":"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e","Type":"ContainerDied","Data":"31dd1f44831cfedb6d1a2a95cdfef25fef46b87f262529c5d8ffc0472c1b6c7a"}
Jul 20 15:51:10.610010 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:10.609982    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:10.686806 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:10.686762    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:10.770121 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:10.770068    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:10.863658 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:10.863562    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:10.863658 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:10.863601    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:10.864136 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:10.863665    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:10.864136 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:10.863681    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t4wh7" podUID="672c6793-5272-43d1-aa5d-ce5bf204f783"
Jul 20 15:51:10.864136 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:10.863772    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hggvm" podUID="b63c3b63-90d2-45eb-ae0d-8491fa36761c"
Jul 20 15:51:10.864136 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:10.863861    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lplvs" podUID="0892d712-0444-4ed9-8da9-41d802a56f8b"
Jul 20 15:51:10.888911 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:10.888873    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:10.951294 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:10.951261    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:11.100942 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.100917    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeReady"
Jul 20 15:51:11.101116 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.101031    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 20 15:51:11.136548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.136469    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"]
Jul 20 15:51:11.138362 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.138336    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.140424 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.140402    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 20 15:51:11.140555 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.140433    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 20 15:51:11.140638 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.140568    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 20 15:51:11.140638 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.140600    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-js84c\""
Jul 20 15:51:11.145957 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.145934    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 20 15:51:11.151524 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.151499    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-btzkj"]
Jul 20 15:51:11.153838 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.153813    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"]
Jul 20 15:51:11.153941 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.153918    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.156042 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.156019    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 20 15:51:11.156042 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.156034    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 20 15:51:11.156201 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.156125    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5lhc8\""
Jul 20 15:51:11.160447 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.160420    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-j5hj8"]
Jul 20 15:51:11.163046 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.163024    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.165814 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.165789    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 20 15:51:11.165915 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.165830    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 20 15:51:11.166259 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.166230    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d2lnj\""
Jul 20 15:51:11.166259 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.166250    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 20 15:51:11.168022 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.168003    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 20 15:51:11.169434 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.169413    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-btzkj"]
Jul 20 15:51:11.176858 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.176834    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j5hj8"]
Jul 20 15:51:11.179211 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.179189    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fpklh"]
Jul 20 15:51:11.181302 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.181281    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fpklh"
Jul 20 15:51:11.185093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.185068    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 20 15:51:11.185336 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.185192    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 20 15:51:11.185336 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.185222    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z5q65\""
Jul 20 15:51:11.185461 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.185412    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 20 15:51:11.194836 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.194815    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fpklh"]
Jul 20 15:51:11.214260 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.214215    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2ace347-d1c8-4286-9541-4885e78bb361-ca-trust-extracted\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.214435 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.214266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-registry-certificates\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.214435 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.214304    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh5gn\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-kube-api-access-dh5gn\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.214435 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.214384    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-image-registry-private-configuration\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.214435 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.214418    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.214926 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.214472    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-bound-sa-token\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.214926 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.214490    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-trusted-ca\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.214926 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.214519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-installation-pull-secrets\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.297608 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.297573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:11.315838 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.315797    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-bound-sa-token\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.315838 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.315838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-trusted-ca\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.316079 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.315866    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlm4n\" (UniqueName: \"kubernetes.io/projected/a085a9d9-de82-432f-aa9e-c82a87d42243-kube-api-access-nlm4n\") pod \"ingress-canary-fpklh\" (UID: \"a085a9d9-de82-432f-aa9e-c82a87d42243\") " pod="openshift-ingress-canary/ingress-canary-fpklh"
Jul 20 15:51:11.316079 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.315899    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-installation-pull-secrets\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.316079 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.315920    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a085a9d9-de82-432f-aa9e-c82a87d42243-cert\") pod \"ingress-canary-fpklh\" (UID: \"a085a9d9-de82-432f-aa9e-c82a87d42243\") " pod="openshift-ingress-canary/ingress-canary-fpklh"
Jul 20 15:51:11.316079 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.315978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2ace347-d1c8-4286-9541-4885e78bb361-ca-trust-extracted\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.316079 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316035    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.316321 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316084    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.316321 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ecd35be-c0a9-4936-8b02-97bf6b446e03-tmp-dir\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.316321 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-registry-certificates\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.316321 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hgwb\" (UniqueName: \"kubernetes.io/projected/70a70572-b756-4cda-9434-35357c5a2024-kube-api-access-2hgwb\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.316321 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh5gn\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-kube-api-access-dh5gn\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.316321 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316230    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrvc4\" (UniqueName: \"kubernetes.io/projected/1ecd35be-c0a9-4936-8b02-97bf6b446e03-kube-api-access-rrvc4\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.316321 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70a70572-b756-4cda-9434-35357c5a2024-crio-socket\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.316321 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316289    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70a70572-b756-4cda-9434-35357c5a2024-data-volume\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.316321 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316314    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.316720 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316338    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ecd35be-c0a9-4936-8b02-97bf6b446e03-config-volume\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.316720 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316383    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-image-registry-private-configuration\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.316720 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316602    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.316853 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2ace347-d1c8-4286-9541-4885e78bb361-ca-trust-extracted\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.316853 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.316755    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 15:51:11.316853 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.316771    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd6c6dcb-fxx4k: secret "image-registry-tls" not found
Jul 20 15:51:11.316853 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.316837    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls podName:f2ace347-d1c8-4286-9541-4885e78bb361 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:11.816817913 +0000 UTC m=+36.472081172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls") pod "image-registry-74dd6c6dcb-fxx4k" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361") : secret "image-registry-tls" not found
Jul 20 15:51:11.317038 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.316946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-registry-certificates\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.317477 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.317183    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-trusted-ca\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.321584 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.321560    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-installation-pull-secrets\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.321714 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.321584    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-image-registry-private-configuration\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.328002 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.327976    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-bound-sa-token\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.328002 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.327986    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh5gn\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-kube-api-access-dh5gn\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.397790 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.397711    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:11.417721 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.417674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70a70572-b756-4cda-9434-35357c5a2024-data-volume\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.417835 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.417724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.417835 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.417766    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ecd35be-c0a9-4936-8b02-97bf6b446e03-config-volume\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.417835 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.417828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlm4n\" (UniqueName: \"kubernetes.io/projected/a085a9d9-de82-432f-aa9e-c82a87d42243-kube-api-access-nlm4n\") pod \"ingress-canary-fpklh\" (UID: \"a085a9d9-de82-432f-aa9e-c82a87d42243\") " pod="openshift-ingress-canary/ingress-canary-fpklh"
Jul 20 15:51:11.417945 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.417850    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a085a9d9-de82-432f-aa9e-c82a87d42243-cert\") pod \"ingress-canary-fpklh\" (UID: \"a085a9d9-de82-432f-aa9e-c82a87d42243\") " pod="openshift-ingress-canary/ingress-canary-fpklh"
Jul 20 15:51:11.417945 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.417901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.417945 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.417931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.418085 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.417952    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ecd35be-c0a9-4936-8b02-97bf6b446e03-tmp-dir\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.418085 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.417983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hgwb\" (UniqueName: \"kubernetes.io/projected/70a70572-b756-4cda-9434-35357c5a2024-kube-api-access-2hgwb\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.418085 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.417999    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 15:51:11.418085 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.418077    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls podName:70a70572-b756-4cda-9434-35357c5a2024 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:11.918056031 +0000 UTC m=+36.573319313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j5hj8" (UID: "70a70572-b756-4cda-9434-35357c5a2024") : secret "insights-runtime-extractor-tls" not found
Jul 20 15:51:11.418289 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.418119    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70a70572-b756-4cda-9434-35357c5a2024-data-volume\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.418289 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.418008    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrvc4\" (UniqueName: \"kubernetes.io/projected/1ecd35be-c0a9-4936-8b02-97bf6b446e03-kube-api-access-rrvc4\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.418289 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.418009    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 15:51:11.418289 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.418183    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70a70572-b756-4cda-9434-35357c5a2024-crio-socket\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.418289 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.418200    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm podName:70a70572-b756-4cda-9434-35357c5a2024 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:11.918182577 +0000 UTC m=+36.573445831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j5hj8" (UID: "70a70572-b756-4cda-9434-35357c5a2024") : configmap "kube-rbac-proxy" not found
Jul 20 15:51:11.418493 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.418306    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 15:51:11.418493 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.418344    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls podName:1ecd35be-c0a9-4936-8b02-97bf6b446e03 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:11.918332655 +0000 UTC m=+36.573595909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls") pod "dns-default-btzkj" (UID: "1ecd35be-c0a9-4936-8b02-97bf6b446e03") : secret "dns-default-metrics-tls" not found
Jul 20 15:51:11.418493 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.418340    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ecd35be-c0a9-4936-8b02-97bf6b446e03-tmp-dir\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.418493 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.418378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70a70572-b756-4cda-9434-35357c5a2024-crio-socket\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.418493 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.418431    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ecd35be-c0a9-4936-8b02-97bf6b446e03-config-volume\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.420313 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.420294    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a085a9d9-de82-432f-aa9e-c82a87d42243-cert\") pod \"ingress-canary-fpklh\" (UID: \"a085a9d9-de82-432f-aa9e-c82a87d42243\") " pod="openshift-ingress-canary/ingress-canary-fpklh"
Jul 20 15:51:11.425989 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.425959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrvc4\" (UniqueName: \"kubernetes.io/projected/1ecd35be-c0a9-4936-8b02-97bf6b446e03-kube-api-access-rrvc4\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.426112 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.426010    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hgwb\" (UniqueName: \"kubernetes.io/projected/70a70572-b756-4cda-9434-35357c5a2024-kube-api-access-2hgwb\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.426173 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.426164    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlm4n\" (UniqueName: \"kubernetes.io/projected/a085a9d9-de82-432f-aa9e-c82a87d42243-kube-api-access-nlm4n\") pod \"ingress-canary-fpklh\" (UID: \"a085a9d9-de82-432f-aa9e-c82a87d42243\") " pod="openshift-ingress-canary/ingress-canary-fpklh"
Jul 20 15:51:11.491608 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.491569    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fpklh"
Jul 20 15:51:11.509414 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.509381    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:11.590603 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.589238    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4lbp5_d0ab2581-3283-415f-b373-d3eedae55bfa/dns-node-resolver/0.log"
Jul 20 15:51:11.612357 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.612327    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:11.653647 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.653573    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fpklh"]
Jul 20 15:51:11.657416 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:11.657389    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda085a9d9_de82_432f_aa9e_c82a87d42243.slice/crio-553dc38093f7cfb46b664906052395fcc33d4d482e812e7a4b4d283c53f13bfe WatchSource:0}: Error finding container 553dc38093f7cfb46b664906052395fcc33d4d482e812e7a4b4d283c53f13bfe: Status 404 returned error can't find the container with id 553dc38093f7cfb46b664906052395fcc33d4d482e812e7a4b4d283c53f13bfe
Jul 20 15:51:11.821254 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.821219    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:11.821414 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.821382    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 15:51:11.821414 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.821399    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd6c6dcb-fxx4k: secret "image-registry-tls" not found
Jul 20 15:51:11.821493 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.821461    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls podName:f2ace347-d1c8-4286-9541-4885e78bb361 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:12.821440682 +0000 UTC m=+37.476703953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls") pod "image-registry-74dd6c6dcb-fxx4k" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361") : secret "image-registry-tls" not found
Jul 20 15:51:11.921869 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.921783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.921869 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.921828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:11.922575 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:11.921871    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:11.922575 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.921939    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 15:51:11.922575 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.921991    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 15:51:11.922575 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.921993    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 15:51:11.922575 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.922003    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls podName:70a70572-b756-4cda-9434-35357c5a2024 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:12.921985544 +0000 UTC m=+37.577248831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j5hj8" (UID: "70a70572-b756-4cda-9434-35357c5a2024") : secret "insights-runtime-extractor-tls" not found
Jul 20 15:51:11.922575 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.922046    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm podName:70a70572-b756-4cda-9434-35357c5a2024 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:12.922032931 +0000 UTC m=+37.577296190 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j5hj8" (UID: "70a70572-b756-4cda-9434-35357c5a2024") : configmap "kube-rbac-proxy" not found
Jul 20 15:51:11.922575 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:11.922062    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls podName:1ecd35be-c0a9-4936-8b02-97bf6b446e03 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:12.922053157 +0000 UTC m=+37.577316417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls") pod "dns-default-btzkj" (UID: "1ecd35be-c0a9-4936-8b02-97bf6b446e03") : secret "dns-default-metrics-tls" not found
Jul 20 15:51:12.093296 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.093257    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fpklh" event={"ID":"a085a9d9-de82-432f-aa9e-c82a87d42243","Type":"ContainerStarted","Data":"553dc38093f7cfb46b664906052395fcc33d4d482e812e7a4b4d283c53f13bfe"}
Jul 20 15:51:12.221656 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.221581    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:12.324413 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.324312    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:12.832006 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.831843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:12.832006 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:12.831972    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 15:51:12.832006 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:12.831996    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd6c6dcb-fxx4k: secret "image-registry-tls" not found
Jul 20 15:51:12.832252 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:12.832066    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls podName:f2ace347-d1c8-4286-9541-4885e78bb361 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:14.832046621 +0000 UTC m=+39.487309880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls") pod "image-registry-74dd6c6dcb-fxx4k" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361") : secret "image-registry-tls" not found
Jul 20 15:51:12.863840 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.863756    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:12.863840 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.863756    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:12.864129 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.863906    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:12.866706 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.866669    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 15:51:12.866835 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.866751    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 15:51:12.867299 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.867195    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6j7cm\""
Jul 20 15:51:12.867477 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.867440    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 15:51:12.867586 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.867477    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 15:51:12.867586 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.867560    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7vt6p\""
Jul 20 15:51:12.933266 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.933229    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:12.933744 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.933280    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:12.933744 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.933331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:12.933744 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:12.933438    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 15:51:12.933744 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:12.933465    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 15:51:12.933744 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:12.933437    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 15:51:12.933744 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:12.933510    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls podName:1ecd35be-c0a9-4936-8b02-97bf6b446e03 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:14.933489095 +0000 UTC m=+39.588752369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls") pod "dns-default-btzkj" (UID: "1ecd35be-c0a9-4936-8b02-97bf6b446e03") : secret "dns-default-metrics-tls" not found
Jul 20 15:51:12.933744 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:12.933529    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm podName:70a70572-b756-4cda-9434-35357c5a2024 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:14.933520157 +0000 UTC m=+39.588783416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j5hj8" (UID: "70a70572-b756-4cda-9434-35357c5a2024") : configmap "kube-rbac-proxy" not found
Jul 20 15:51:12.933744 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:12.933545    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls podName:70a70572-b756-4cda-9434-35357c5a2024 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:14.9335356 +0000 UTC m=+39.588798860 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j5hj8" (UID: "70a70572-b756-4cda-9434-35357c5a2024") : secret "insights-runtime-extractor-tls" not found
Jul 20 15:51:12.984125 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:12.984098    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dvh7c_a518917c-09da-43c3-b61e-f17a704fa9a2/node-ca/0.log"
Jul 20 15:51:13.103736 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.103589    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-dw6ct"]
Jul 20 15:51:13.107090 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.107053    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:13.110222 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.110193    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 20 15:51:13.110367 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.110228    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 20 15:51:13.110367 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.110324    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 20 15:51:13.110367 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.110202    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7vg89\""
Jul 20 15:51:13.110533 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.110500    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 20 15:51:13.112154 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.112103    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-dw6ct"]
Jul 20 15:51:13.131121 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.131091    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:13.236399 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.236367    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/721d29cb-6c98-4c4a-9a63-4de35e690ee8-signing-cabundle\") pod \"service-ca-58598695f7-dw6ct\" (UID: \"721d29cb-6c98-4c4a-9a63-4de35e690ee8\") " pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:13.236568 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.236463    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/721d29cb-6c98-4c4a-9a63-4de35e690ee8-signing-key\") pod \"service-ca-58598695f7-dw6ct\" (UID: \"721d29cb-6c98-4c4a-9a63-4de35e690ee8\") " pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:13.236568 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.236497    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjxzn\" (UniqueName: \"kubernetes.io/projected/721d29cb-6c98-4c4a-9a63-4de35e690ee8-kube-api-access-xjxzn\") pod \"service-ca-58598695f7-dw6ct\" (UID: \"721d29cb-6c98-4c4a-9a63-4de35e690ee8\") " pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:13.337614 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.337574    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/721d29cb-6c98-4c4a-9a63-4de35e690ee8-signing-cabundle\") pod \"service-ca-58598695f7-dw6ct\" (UID: \"721d29cb-6c98-4c4a-9a63-4de35e690ee8\") " pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:13.337814 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.337674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/721d29cb-6c98-4c4a-9a63-4de35e690ee8-signing-key\") pod \"service-ca-58598695f7-dw6ct\" (UID: \"721d29cb-6c98-4c4a-9a63-4de35e690ee8\") " pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:13.337814 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.337729    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjxzn\" (UniqueName: \"kubernetes.io/projected/721d29cb-6c98-4c4a-9a63-4de35e690ee8-kube-api-access-xjxzn\") pod \"service-ca-58598695f7-dw6ct\" (UID: \"721d29cb-6c98-4c4a-9a63-4de35e690ee8\") " pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:13.338504 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.338474    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/721d29cb-6c98-4c4a-9a63-4de35e690ee8-signing-cabundle\") pod \"service-ca-58598695f7-dw6ct\" (UID: \"721d29cb-6c98-4c4a-9a63-4de35e690ee8\") " pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:13.340880 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.340854    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/721d29cb-6c98-4c4a-9a63-4de35e690ee8-signing-key\") pod \"service-ca-58598695f7-dw6ct\" (UID: \"721d29cb-6c98-4c4a-9a63-4de35e690ee8\") " pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:13.346846 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.346819    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjxzn\" (UniqueName: \"kubernetes.io/projected/721d29cb-6c98-4c4a-9a63-4de35e690ee8-kube-api-access-xjxzn\") pod \"service-ca-58598695f7-dw6ct\" (UID: \"721d29cb-6c98-4c4a-9a63-4de35e690ee8\") " pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:13.421387 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:13.421300    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-dw6ct"
Jul 20 15:51:14.104093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:14.104051    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:14.852824 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:14.852577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:14.853011 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:14.852758    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 15:51:14.853011 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:14.852893    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd6c6dcb-fxx4k: secret "image-registry-tls" not found
Jul 20 15:51:14.853011 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:14.852949    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls podName:f2ace347-d1c8-4286-9541-4885e78bb361 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:18.852933149 +0000 UTC m=+43.508196418 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls") pod "image-registry-74dd6c6dcb-fxx4k" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361") : secret "image-registry-tls" not found
Jul 20 15:51:14.953680 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:14.953646    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:14.953868 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:14.953732    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:14.953868 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:14.953814    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 15:51:14.953868 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:14.953849    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:14.954025 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:14.953880    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls podName:1ecd35be-c0a9-4936-8b02-97bf6b446e03 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:18.953860683 +0000 UTC m=+43.609123948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls") pod "dns-default-btzkj" (UID: "1ecd35be-c0a9-4936-8b02-97bf6b446e03") : secret "dns-default-metrics-tls" not found
Jul 20 15:51:14.954025 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:14.953923    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 15:51:14.954025 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:14.953956    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls podName:70a70572-b756-4cda-9434-35357c5a2024 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:18.953945858 +0000 UTC m=+43.609209129 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j5hj8" (UID: "70a70572-b756-4cda-9434-35357c5a2024") : secret "insights-runtime-extractor-tls" not found
Jul 20 15:51:14.954025 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:14.954000    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 15:51:14.954025 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:14.954029    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm podName:70a70572-b756-4cda-9434-35357c5a2024 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:18.954019957 +0000 UTC m=+43.609283225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j5hj8" (UID: "70a70572-b756-4cda-9434-35357c5a2024") : configmap "kube-rbac-proxy" not found
Jul 20 15:51:15.407418 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:15.407384    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:15.674329 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:15.674243    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:15.744640 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:15.744607    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-dw6ct"]
Jul 20 15:51:15.747896 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:15.747874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:15.748349 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:15.748328    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod721d29cb_6c98_4c4a_9a63_4de35e690ee8.slice/crio-a19882ea4076a2161c614eb8b2e8a4fa2e9e1b31456ebfc8a8fa547a5802abff WatchSource:0}: Error finding container a19882ea4076a2161c614eb8b2e8a4fa2e9e1b31456ebfc8a8fa547a5802abff: Status 404 returned error can't find the container with id a19882ea4076a2161c614eb8b2e8a4fa2e9e1b31456ebfc8a8fa547a5802abff
Jul 20 15:51:16.105508 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:16.105472    2564 generic.go:356] "Generic (PLEG): container finished" podID="a46171f5-26e9-4fda-9f5d-6e1ef9a0995e" containerID="e5d6ee50fb71171cfbea16b2584b9716b196c687789ec022776f4799970eb262" exitCode=0
Jul 20 15:51:16.105666 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:16.105562    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8p6lq" event={"ID":"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e","Type":"ContainerDied","Data":"e5d6ee50fb71171cfbea16b2584b9716b196c687789ec022776f4799970eb262"}
Jul 20 15:51:16.106741 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:16.106718    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-dw6ct" event={"ID":"721d29cb-6c98-4c4a-9a63-4de35e690ee8","Type":"ContainerStarted","Data":"a19882ea4076a2161c614eb8b2e8a4fa2e9e1b31456ebfc8a8fa547a5802abff"}
Jul 20 15:51:16.109441 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:16.109408    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fpklh" event={"ID":"a085a9d9-de82-432f-aa9e-c82a87d42243","Type":"ContainerStarted","Data":"09ac81cc484046daa4e703c9d44556ddf01ff6298e642a3a3bf62cdb569de607"}
Jul 20 15:51:16.146470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:16.146421    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fpklh" podStartSLOduration=1.193028815 podStartE2EDuration="5.146403115s" totalImagesPullingTime="3.9533743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:16.145776631 +0000 UTC m=+40.801039907" watchObservedRunningTime="2026-07-20 15:51:16.146403115 +0000 UTC m=+40.801666392"
Jul 20 15:51:17.115000 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:17.114969    2564 generic.go:356] "Generic (PLEG): container finished" podID="a46171f5-26e9-4fda-9f5d-6e1ef9a0995e" containerID="90777761939ab63f9f92a00ab9017b1cc7feecfceec3db87d3b0952e7f344176" exitCode=0
Jul 20 15:51:17.115382 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:17.115032    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8p6lq" event={"ID":"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e","Type":"ContainerDied","Data":"90777761939ab63f9f92a00ab9017b1cc7feecfceec3db87d3b0952e7f344176"}
Jul 20 15:51:17.166526 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:17.166495    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:18.121165 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:18.121006    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8p6lq" event={"ID":"a46171f5-26e9-4fda-9f5d-6e1ef9a0995e","Type":"ContainerStarted","Data":"c1dd6bbee86e52be7ae6340d29236495fd9933eb88029e761970752b1da28cab"}
Jul 20 15:51:18.150331 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:18.150263    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8p6lq" podStartSLOduration=10.870679248 podStartE2EDuration="43.15023929s" totalImagesPullingTime="32.279560042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:18.149480478 +0000 UTC m=+42.804743762" watchObservedRunningTime="2026-07-20 15:51:18.15023929 +0000 UTC m=+42.805502566"
Jul 20 15:51:18.892951 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:18.892911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:18.893114 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:18.893066    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 15:51:18.893114 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:18.893087    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd6c6dcb-fxx4k: secret "image-registry-tls" not found
Jul 20 15:51:18.893195 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:18.893143    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls podName:f2ace347-d1c8-4286-9541-4885e78bb361 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:26.893127135 +0000 UTC m=+51.548390392 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls") pod "image-registry-74dd6c6dcb-fxx4k" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361") : secret "image-registry-tls" not found
Jul 20 15:51:18.994111 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:18.994067    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:18.994111 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:18.994111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:18.994313 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:18.994142    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:18.994313 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:18.994218    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 15:51:18.994313 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:18.994231    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 15:51:18.994313 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:18.994285    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls podName:70a70572-b756-4cda-9434-35357c5a2024 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:26.994268238 +0000 UTC m=+51.649531494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j5hj8" (UID: "70a70572-b756-4cda-9434-35357c5a2024") : secret "insights-runtime-extractor-tls" not found
Jul 20 15:51:18.994313 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:51:18.994314    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls podName:1ecd35be-c0a9-4936-8b02-97bf6b446e03 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:26.994307263 +0000 UTC m=+51.649570516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls") pod "dns-default-btzkj" (UID: "1ecd35be-c0a9-4936-8b02-97bf6b446e03") : secret "dns-default-metrics-tls" not found
Jul 20 15:51:18.994681 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:18.994665    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70a70572-b756-4cda-9434-35357c5a2024-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:19.127889 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:19.127844    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-dw6ct" event={"ID":"721d29cb-6c98-4c4a-9a63-4de35e690ee8","Type":"ContainerStarted","Data":"fb490b2d71f4f6429d4fc54f0fa4bd04a76d9e074b62f116d62295ec8f8a58a6"}
Jul 20 15:51:19.145316 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:19.145212    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-dw6ct" podStartSLOduration=3.8249632780000002 podStartE2EDuration="6.145196855s" totalImagesPullingTime="2.320233577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:19.143936317 +0000 UTC m=+43.799199594" watchObservedRunningTime="2026-07-20 15:51:19.145196855 +0000 UTC m=+43.800460172"
Jul 20 15:51:20.003809 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:20.003772    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:20.101846 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:20.101805    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:20.200978 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:20.200938    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:20.302398 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:20.302364    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:20.411184 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:20.411146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:21.100089 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:21.100052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:21.700450 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:21.700408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:21.901907 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:21.901878    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:22.294899 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:22.294863    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:22.501511 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:22.501479    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:22.711098 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:22.711006    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:23.100180 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:23.100145    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:23.697352 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:23.697320    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:23.750313 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:23.750276    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:23.903154 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:23.903124    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:24.300305 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:24.300267    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:24.402095 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:24.402067    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:24.501472 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:24.501437    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:24.602707 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:24.602612    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:24.652496 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:24.652462    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:24.912028 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:24.911945    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:25.498627 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:25.498592    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:25.602221 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:25.602186    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:25.899219 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:25.899185    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:25.952330 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:25.952298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:26.297853 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:26.297823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:26.501017 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:26.500980    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:26.899887 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:26.899851    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:26.959042 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:26.959007    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:26.962096 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:26.962075    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls\") pod \"image-registry-74dd6c6dcb-fxx4k\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") " pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:27.050071 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.050032    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:27.060375 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.060341    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:27.060375 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.060375    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:27.062879 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.062849    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ecd35be-c0a9-4936-8b02-97bf6b446e03-metrics-tls\") pod \"dns-default-btzkj\" (UID: \"1ecd35be-c0a9-4936-8b02-97bf6b446e03\") " pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:27.063008 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.062971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70a70572-b756-4cda-9434-35357c5a2024-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j5hj8\" (UID: \"70a70572-b756-4cda-9434-35357c5a2024\") " pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:27.069884 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.069859    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:27.076552 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.076530    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j5hj8"
Jul 20 15:51:27.216341 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.216304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:27.220588 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.220541    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"]
Jul 20 15:51:27.224791 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:27.224750    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ace347_d1c8_4286_9541_4885e78bb361.slice/crio-0891d7ffd9e7469fd064ac0568655595a236151883301d9120794ddc65247d99 WatchSource:0}: Error finding container 0891d7ffd9e7469fd064ac0568655595a236151883301d9120794ddc65247d99: Status 404 returned error can't find the container with id 0891d7ffd9e7469fd064ac0568655595a236151883301d9120794ddc65247d99
Jul 20 15:51:27.243922 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.243888    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-btzkj"]
Jul 20 15:51:27.247141 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:27.247103    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ecd35be_c0a9_4936_8b02_97bf6b446e03.slice/crio-43aed356e277c0a9b9a363719a94c64fa87f1a3d6d9b7dc3fd2fa80ebbf0bb04 WatchSource:0}: Error finding container 43aed356e277c0a9b9a363719a94c64fa87f1a3d6d9b7dc3fd2fa80ebbf0bb04: Status 404 returned error can't find the container with id 43aed356e277c0a9b9a363719a94c64fa87f1a3d6d9b7dc3fd2fa80ebbf0bb04
Jul 20 15:51:27.262964 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.262933    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j5hj8"]
Jul 20 15:51:27.281778 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:27.281745    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70a70572_b756_4cda_9434_35357c5a2024.slice/crio-f5a6dbced57feda25f61c7714e4f12628383bb4effad9aba2daf15fc5096e332 WatchSource:0}: Error finding container f5a6dbced57feda25f61c7714e4f12628383bb4effad9aba2daf15fc5096e332: Status 404 returned error can't find the container with id f5a6dbced57feda25f61c7714e4f12628383bb4effad9aba2daf15fc5096e332
Jul 20 15:51:27.317562 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.317538    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:27.504103 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:27.503967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:28.109453 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:28.109383    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:28.154242 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:28.154158    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k" event={"ID":"f2ace347-d1c8-4286-9541-4885e78bb361","Type":"ContainerStarted","Data":"79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077"}
Jul 20 15:51:28.155201 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:28.154218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k" event={"ID":"f2ace347-d1c8-4286-9541-4885e78bb361","Type":"ContainerStarted","Data":"0891d7ffd9e7469fd064ac0568655595a236151883301d9120794ddc65247d99"}
Jul 20 15:51:28.155201 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:28.155159    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:28.157931 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:28.157904    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j5hj8" event={"ID":"70a70572-b756-4cda-9434-35357c5a2024","Type":"ContainerStarted","Data":"f5a6dbced57feda25f61c7714e4f12628383bb4effad9aba2daf15fc5096e332"}
Jul 20 15:51:28.161035 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:28.160939    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-btzkj" event={"ID":"1ecd35be-c0a9-4936-8b02-97bf6b446e03","Type":"ContainerStarted","Data":"43aed356e277c0a9b9a363719a94c64fa87f1a3d6d9b7dc3fd2fa80ebbf0bb04"}
Jul 20 15:51:28.175391 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:28.174608    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k" podStartSLOduration=28.174570893 podStartE2EDuration="28.174570893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:28.173217703 +0000 UTC m=+52.828480981" watchObservedRunningTime="2026-07-20 15:51:28.174570893 +0000 UTC m=+52.829834170"
Jul 20 15:51:28.302777 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:28.302669    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:28.701699 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:28.701608    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:28.907759 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:28.907713    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:29.305494 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:29.305461    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:29.896948 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:29.896860    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:30.168711 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:30.168591    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-btzkj" event={"ID":"1ecd35be-c0a9-4936-8b02-97bf6b446e03","Type":"ContainerStarted","Data":"6aaac6fe92faa973a06a5a9782798a3a2490aa44c70b35e3e5ae00a151fa3600"}
Jul 20 15:51:30.429676 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:30.429583    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:30.530978 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:30.530945    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:30.700637 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:30.700552    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:33.179989 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:33.179949    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j5hj8" event={"ID":"70a70572-b756-4cda-9434-35357c5a2024","Type":"ContainerStarted","Data":"2cceed25cd273b379411e3e061aa452095cb4079f18645c4c623304200e7d8a4"}
Jul 20 15:51:33.181543 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:33.181518    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-btzkj" event={"ID":"1ecd35be-c0a9-4936-8b02-97bf6b446e03","Type":"ContainerStarted","Data":"fe84cc165363955b5a26e61fac0900c8a6aa94251ce4d72de66e05573d900327"}
Jul 20 15:51:33.181674 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:33.181662    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:33.197427 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:33.197374    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-btzkj" podStartSLOduration=19.622836165 podStartE2EDuration="22.197356243s" totalImagesPullingTime="2.574520078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:33.196715658 +0000 UTC m=+57.851978936" watchObservedRunningTime="2026-07-20 15:51:33.197356243 +0000 UTC m=+57.852619519"
Jul 20 15:51:35.535846 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.533967    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"]
Jul 20 15:51:35.541958 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.541766    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"]
Jul 20 15:51:35.541958 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.541902    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.543635 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.543616    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"]
Jul 20 15:51:35.543857 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.543768    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"
Jul 20 15:51:35.544582 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.544375    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 20 15:51:35.544582 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.544468    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 20 15:51:35.544882 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.544838    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 20 15:51:35.545029 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.545007    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 20 15:51:35.545843 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.545823    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.553583 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.553562    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 20 15:51:35.554141 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.554119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-h7lkl\""
Jul 20 15:51:35.554253 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.554144    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 20 15:51:35.554253 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.554119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 20 15:51:35.554253 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.554122    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 20 15:51:35.554404 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.554241    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 20 15:51:35.581887 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.581815    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"]
Jul 20 15:51:35.583165 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.583141    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"]
Jul 20 15:51:35.583722 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.583668    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"]
Jul 20 15:51:35.583915 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.583857    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:35.651568 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.651530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-hub\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.651568 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.651572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-ca\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.651804 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.651599    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvdx4\" (UniqueName: \"kubernetes.io/projected/1e680896-a759-4ec2-9619-9b80b483a558-kube-api-access-gvdx4\") pod \"managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg\" (UID: \"1e680896-a759-4ec2-9619-9b80b483a558\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"
Jul 20 15:51:35.651804 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.651755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f35eea9b-d385-4060-afde-7b24e22cbda6-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2\" (UID: \"f35eea9b-d385-4060-afde-7b24e22cbda6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.651804 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.651789    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.651920 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.651812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.651920 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.651856    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87bwx\" (UniqueName: \"kubernetes.io/projected/f35eea9b-d385-4060-afde-7b24e22cbda6-kube-api-access-87bwx\") pod \"klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2\" (UID: \"f35eea9b-d385-4060-afde-7b24e22cbda6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.652003 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.651929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35eea9b-d385-4060-afde-7b24e22cbda6-tmp\") pod \"klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2\" (UID: \"f35eea9b-d385-4060-afde-7b24e22cbda6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.652003 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.651949    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f65cba-b2e7-4d1b-b943-c879e3175d9c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.652003 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.651973    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbjdp\" (UniqueName: \"kubernetes.io/projected/72f65cba-b2e7-4d1b-b943-c879e3175d9c-kube-api-access-hbjdp\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.652147 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.652087    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e680896-a759-4ec2-9619-9b80b483a558-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg\" (UID: \"1e680896-a759-4ec2-9619-9b80b483a558\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"
Jul 20 15:51:35.729294 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.729263    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"]
Jul 20 15:51:35.736538 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.736501    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:35.753131 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f35eea9b-d385-4060-afde-7b24e22cbda6-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2\" (UID: \"f35eea9b-d385-4060-afde-7b24e22cbda6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.753131 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753135    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.753373 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.753373 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87bwx\" (UniqueName: \"kubernetes.io/projected/f35eea9b-d385-4060-afde-7b24e22cbda6-kube-api-access-87bwx\") pod \"klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2\" (UID: \"f35eea9b-d385-4060-afde-7b24e22cbda6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.753373 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35eea9b-d385-4060-afde-7b24e22cbda6-tmp\") pod \"klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2\" (UID: \"f35eea9b-d385-4060-afde-7b24e22cbda6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.753533 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f65cba-b2e7-4d1b-b943-c879e3175d9c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.753533 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753423    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbjdp\" (UniqueName: \"kubernetes.io/projected/72f65cba-b2e7-4d1b-b943-c879e3175d9c-kube-api-access-hbjdp\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.753533 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753468    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e680896-a759-4ec2-9619-9b80b483a558-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg\" (UID: \"1e680896-a759-4ec2-9619-9b80b483a558\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"
Jul 20 15:51:35.753533 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-hub\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.753744 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-ca\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.753744 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753593    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvdx4\" (UniqueName: \"kubernetes.io/projected/1e680896-a759-4ec2-9619-9b80b483a558-kube-api-access-gvdx4\") pod \"managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg\" (UID: \"1e680896-a759-4ec2-9619-9b80b483a558\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"
Jul 20 15:51:35.753744 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.753700    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35eea9b-d385-4060-afde-7b24e22cbda6-tmp\") pod \"klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2\" (UID: \"f35eea9b-d385-4060-afde-7b24e22cbda6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.755822 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.755774    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 20 15:51:35.755952 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.755840    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 20 15:51:35.755952 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.755919    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 20 15:51:35.755952 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.755927    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 20 15:51:35.756307 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.756288    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 20 15:51:35.756307 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.756307    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 20 15:51:35.756448 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.756317    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 20 15:51:35.762736 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.762714    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 20 15:51:35.765099 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.765022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f65cba-b2e7-4d1b-b943-c879e3175d9c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.767158 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.767128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.767278 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.767175    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e680896-a759-4ec2-9619-9b80b483a558-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg\" (UID: \"1e680896-a759-4ec2-9619-9b80b483a558\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"
Jul 20 15:51:35.767278 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.767233    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f35eea9b-d385-4060-afde-7b24e22cbda6-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2\" (UID: \"f35eea9b-d385-4060-afde-7b24e22cbda6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.768301 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.768262    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.768927 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.768904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-hub\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.769013 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.768914    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/72f65cba-b2e7-4d1b-b943-c879e3175d9c-ca\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.772807 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.772780    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvdx4\" (UniqueName: \"kubernetes.io/projected/1e680896-a759-4ec2-9619-9b80b483a558-kube-api-access-gvdx4\") pod \"managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg\" (UID: \"1e680896-a759-4ec2-9619-9b80b483a558\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"
Jul 20 15:51:35.773936 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.773913    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87bwx\" (UniqueName: \"kubernetes.io/projected/f35eea9b-d385-4060-afde-7b24e22cbda6-kube-api-access-87bwx\") pod \"klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2\" (UID: \"f35eea9b-d385-4060-afde-7b24e22cbda6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.775425 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.775399    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbjdp\" (UniqueName: \"kubernetes.io/projected/72f65cba-b2e7-4d1b-b943-c879e3175d9c-kube-api-access-hbjdp\") pod \"cluster-proxy-proxy-agent-86475b7db4-tnph6\" (UID: \"72f65cba-b2e7-4d1b-b943-c879e3175d9c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.783188 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.783155    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69d78676d4-mds9q"]
Jul 20 15:51:35.790644 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.790569    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.800500 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.800473    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69d78676d4-mds9q"]
Jul 20 15:51:35.830610 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.830566    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:35.854017 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.853982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ecc6170f-e8c0-4ac4-ac53-729123ea019b-installation-pull-secrets\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.854203 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.854069    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ecc6170f-e8c0-4ac4-ac53-729123ea019b-registry-tls\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.854203 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.854193    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecc6170f-e8c0-4ac4-ac53-729123ea019b-bound-sa-token\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.854331 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.854226    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hpv7\" (UniqueName: \"kubernetes.io/projected/ecc6170f-e8c0-4ac4-ac53-729123ea019b-kube-api-access-9hpv7\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.854331 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.854256    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecc6170f-e8c0-4ac4-ac53-729123ea019b-trusted-ca\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.854417 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.854335    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ecc6170f-e8c0-4ac4-ac53-729123ea019b-ca-trust-extracted\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.854417 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.854371    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ecc6170f-e8c0-4ac4-ac53-729123ea019b-registry-certificates\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.854417 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.854409    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ecc6170f-e8c0-4ac4-ac53-729123ea019b-image-registry-private-configuration\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.855667 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.855580    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 20 15:51:35.863569 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.863531    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:35.881837 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.881807    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-h7lkl\""
Jul 20 15:51:35.886171 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.886142    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"
Jul 20 15:51:35.888934 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.888900    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"
Jul 20 15:51:35.944479 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.944443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:35.955218 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.955189    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ecc6170f-e8c0-4ac4-ac53-729123ea019b-installation-pull-secrets\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.955406 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.955249    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ecc6170f-e8c0-4ac4-ac53-729123ea019b-registry-tls\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.955406 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.955318    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecc6170f-e8c0-4ac4-ac53-729123ea019b-bound-sa-token\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.955406 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.955340    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hpv7\" (UniqueName: \"kubernetes.io/projected/ecc6170f-e8c0-4ac4-ac53-729123ea019b-kube-api-access-9hpv7\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.955406 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.955365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecc6170f-e8c0-4ac4-ac53-729123ea019b-trusted-ca\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.955406 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.955396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ecc6170f-e8c0-4ac4-ac53-729123ea019b-ca-trust-extracted\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.955668 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.955571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ecc6170f-e8c0-4ac4-ac53-729123ea019b-registry-certificates\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.955668 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.955625    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ecc6170f-e8c0-4ac4-ac53-729123ea019b-image-registry-private-configuration\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.956397 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.956368    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ecc6170f-e8c0-4ac4-ac53-729123ea019b-ca-trust-extracted\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.956950 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.956905    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecc6170f-e8c0-4ac4-ac53-729123ea019b-trusted-ca\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.957053 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.956995    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ecc6170f-e8c0-4ac4-ac53-729123ea019b-registry-certificates\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.958304 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.958283    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ecc6170f-e8c0-4ac4-ac53-729123ea019b-image-registry-private-configuration\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.958396 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.958333    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ecc6170f-e8c0-4ac4-ac53-729123ea019b-installation-pull-secrets\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.958861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.958841    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ecc6170f-e8c0-4ac4-ac53-729123ea019b-registry-tls\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.964839 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.964815    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecc6170f-e8c0-4ac4-ac53-729123ea019b-bound-sa-token\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:35.965143 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:35.965102    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hpv7\" (UniqueName: \"kubernetes.io/projected/ecc6170f-e8c0-4ac4-ac53-729123ea019b-kube-api-access-9hpv7\") pod \"image-registry-69d78676d4-mds9q\" (UID: \"ecc6170f-e8c0-4ac4-ac53-729123ea019b\") " pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:36.041736 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:36.041644    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:36.110294 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:36.110263    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:36.143944 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:36.143903    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:36.330638 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:36.330600    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg"]
Jul 20 15:51:36.339740 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:36.339709    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e680896_a759_4ec2_9619_9b80b483a558.slice/crio-11c56c536fc178ba853fea06f19a0d2c9da26b5662c13fb8a225b598226c9996 WatchSource:0}: Error finding container 11c56c536fc178ba853fea06f19a0d2c9da26b5662c13fb8a225b598226c9996: Status 404 returned error can't find the container with id 11c56c536fc178ba853fea06f19a0d2c9da26b5662c13fb8a225b598226c9996
Jul 20 15:51:36.351299 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:36.351271    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6"]
Jul 20 15:51:36.354055 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:36.354029    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72f65cba_b2e7_4d1b_b943_c879e3175d9c.slice/crio-6b5ce5f37f2315b4ae393fd03bf245a61c43a071dc83371d6a7bc90f426bb154 WatchSource:0}: Error finding container 6b5ce5f37f2315b4ae393fd03bf245a61c43a071dc83371d6a7bc90f426bb154: Status 404 returned error can't find the container with id 6b5ce5f37f2315b4ae393fd03bf245a61c43a071dc83371d6a7bc90f426bb154
Jul 20 15:51:36.562132 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:36.562020    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"]
Jul 20 15:51:36.565026 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:36.565002    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69d78676d4-mds9q"]
Jul 20 15:51:36.566256 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:36.566232    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35eea9b_d385_4060_afde_7b24e22cbda6.slice/crio-0e06496cf1061957a264bb7f2b95fd06d84634519f08eaa15b67ec41a3e8f1a1 WatchSource:0}: Error finding container 0e06496cf1061957a264bb7f2b95fd06d84634519f08eaa15b67ec41a3e8f1a1: Status 404 returned error can't find the container with id 0e06496cf1061957a264bb7f2b95fd06d84634519f08eaa15b67ec41a3e8f1a1
Jul 20 15:51:36.567557 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:36.567519    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc6170f_e8c0_4ac4_ac53_729123ea019b.slice/crio-0f71578b8f9b9242339fece2a65f0bcfdc52a020840b11b4c0c95c87e35b28b7 WatchSource:0}: Error finding container 0f71578b8f9b9242339fece2a65f0bcfdc52a020840b11b4c0c95c87e35b28b7: Status 404 returned error can't find the container with id 0f71578b8f9b9242339fece2a65f0bcfdc52a020840b11b4c0c95c87e35b28b7
Jul 20 15:51:37.213233 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:37.213129    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6" event={"ID":"72f65cba-b2e7-4d1b-b943-c879e3175d9c","Type":"ContainerStarted","Data":"6b5ce5f37f2315b4ae393fd03bf245a61c43a071dc83371d6a7bc90f426bb154"}
Jul 20 15:51:37.216642 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:37.216601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j5hj8" event={"ID":"70a70572-b756-4cda-9434-35357c5a2024","Type":"ContainerStarted","Data":"b14d149aa095220c732a43d85af2cdb2197b1e91a87572815e385a22c6d3c500"}
Jul 20 15:51:37.218613 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:37.218584    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg" event={"ID":"1e680896-a759-4ec2-9619-9b80b483a558","Type":"ContainerStarted","Data":"11c56c536fc178ba853fea06f19a0d2c9da26b5662c13fb8a225b598226c9996"}
Jul 20 15:51:37.221510 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:37.221455    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2" event={"ID":"f35eea9b-d385-4060-afde-7b24e22cbda6","Type":"ContainerStarted","Data":"0e06496cf1061957a264bb7f2b95fd06d84634519f08eaa15b67ec41a3e8f1a1"}
Jul 20 15:51:37.224708 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:37.224662    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d78676d4-mds9q" event={"ID":"ecc6170f-e8c0-4ac4-ac53-729123ea019b","Type":"ContainerStarted","Data":"31f00bdac2e6e0c015bbb5f9d17848859140468813dda01324515c7921f644b7"}
Jul 20 15:51:37.224834 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:37.224723    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d78676d4-mds9q" event={"ID":"ecc6170f-e8c0-4ac4-ac53-729123ea019b","Type":"ContainerStarted","Data":"0f71578b8f9b9242339fece2a65f0bcfdc52a020840b11b4c0c95c87e35b28b7"}
Jul 20 15:51:37.225464 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:37.225442    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:37.246482 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:37.246242    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69d78676d4-mds9q" podStartSLOduration=2.246219906 podStartE2EDuration="2.246219906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:37.244616402 +0000 UTC m=+61.899879678" watchObservedRunningTime="2026-07-20 15:51:37.246219906 +0000 UTC m=+61.901483181"
Jul 20 15:51:37.667168 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:37.667094    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:37.726910 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:37.726879    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:39.099487 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:39.099459    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxp2n"
Jul 20 15:51:39.257311 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:39.257276    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:39.317201 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:39.317133    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:39.799394 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:39.799354    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:39.895744 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:39.895682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:39.970536 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:39.970478    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:40.187910 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:40.187827    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-btzkj"
Jul 20 15:51:40.249950 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:40.249910    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:41.414114 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.414076    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:41.811753 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.811706    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:41.811963 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.811769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:41.814362 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.814339    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 15:51:41.814543 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.814525    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 15:51:41.824724 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.824667    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/672c6793-5272-43d1-aa5d-ce5bf204f783-original-pull-secret\") pod \"global-pull-secret-syncer-t4wh7\" (UID: \"672c6793-5272-43d1-aa5d-ce5bf204f783\") " pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:41.824860 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.824668    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0892d712-0444-4ed9-8da9-41d802a56f8b-metrics-certs\") pod \"network-metrics-daemon-lplvs\" (UID: \"0892d712-0444-4ed9-8da9-41d802a56f8b\") " pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:41.912949 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.912912    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhckn\" (UniqueName: \"kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn\") pod \"network-check-target-hggvm\" (UID: \"b63c3b63-90d2-45eb-ae0d-8491fa36761c\") " pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:41.915282 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.915254    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 15:51:41.925846 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.925816    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 15:51:41.937706 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.937661    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhckn\" (UniqueName: \"kubernetes.io/projected/b63c3b63-90d2-45eb-ae0d-8491fa36761c-kube-api-access-lhckn\") pod \"network-check-target-hggvm\" (UID: \"b63c3b63-90d2-45eb-ae0d-8491fa36761c\") " pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:41.980810 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.980777    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7vt6p\""
Jul 20 15:51:41.986955 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.986927    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t4wh7"
Jul 20 15:51:41.989040 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.989021    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lplvs"
Jul 20 15:51:41.996364 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:41.996343    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6j7cm\""
Jul 20 15:51:42.005073 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:42.005048    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:42.997897 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:42.997857    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:43.829306 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:43.829275    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:43.929087 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:43.929050    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:44.261284 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.261237    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j5hj8" event={"ID":"70a70572-b756-4cda-9434-35357c5a2024","Type":"ContainerStarted","Data":"43b14c385145e33259778c043fedbdaf047880cb161cefeef9392c4527ed28ca"}
Jul 20 15:51:44.266017 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.265922    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg" event={"ID":"1e680896-a759-4ec2-9619-9b80b483a558","Type":"ContainerStarted","Data":"bce9dccf9a464f2958c2506d640caaab9486a041776768600297d1cd62b35dfd"}
Jul 20 15:51:44.269381 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.268333    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t4wh7"]
Jul 20 15:51:44.277085 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.273166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2" event={"ID":"f35eea9b-d385-4060-afde-7b24e22cbda6","Type":"ContainerStarted","Data":"1ee64f9b3aaaff519518599010c5cd2109a6455036da29fedacb61be7411d564"}
Jul 20 15:51:44.277085 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.274599    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:44.277085 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.275004    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2" podUID="f35eea9b-d385-4060-afde-7b24e22cbda6" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.11:8000/readyz\": dial tcp 10.133.0.11:8000: connect: connection refused"
Jul 20 15:51:44.277085 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:44.275924    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod672c6793_5272_43d1_aa5d_ce5bf204f783.slice/crio-1ef922c7c56beb0b9190e0ee8e21ae59c47fa23787f4f3eff2cc00409d5b1dfa WatchSource:0}: Error finding container 1ef922c7c56beb0b9190e0ee8e21ae59c47fa23787f4f3eff2cc00409d5b1dfa: Status 404 returned error can't find the container with id 1ef922c7c56beb0b9190e0ee8e21ae59c47fa23787f4f3eff2cc00409d5b1dfa
Jul 20 15:51:44.284557 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.284426    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-j5hj8" podStartSLOduration=16.641054865 podStartE2EDuration="33.284400494s" totalImagesPullingTime="16.643345629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:44.282958602 +0000 UTC m=+68.938221882" watchObservedRunningTime="2026-07-20 15:51:44.284400494 +0000 UTC m=+68.939663771"
Jul 20 15:51:44.297439 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.297407    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hggvm"]
Jul 20 15:51:44.301759 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.301705    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2" podStartSLOduration=1.7870875179999999 podStartE2EDuration="9.301667484s" totalImagesPullingTime="7.514579966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:44.299773687 +0000 UTC m=+68.955036962" watchObservedRunningTime="2026-07-20 15:51:44.301667484 +0000 UTC m=+68.956930761"
Jul 20 15:51:44.303584 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:44.303548    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb63c3b63_90d2_45eb_ae0d_8491fa36761c.slice/crio-e8844e34b15fd3d209a0f2c2ff96cecf9c0ab2847436c173d27447b12dd93fab WatchSource:0}: Error finding container e8844e34b15fd3d209a0f2c2ff96cecf9c0ab2847436c173d27447b12dd93fab: Status 404 returned error can't find the container with id e8844e34b15fd3d209a0f2c2ff96cecf9c0ab2847436c173d27447b12dd93fab
Jul 20 15:51:44.312585 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.312555    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lplvs"]
Jul 20 15:51:44.317034 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.316992    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6c9cf7cb-4l6sg" podStartSLOduration=1.591074083 podStartE2EDuration="9.316972373s" totalImagesPullingTime="7.72589829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:44.316084451 +0000 UTC m=+68.971347727" watchObservedRunningTime="2026-07-20 15:51:44.316972373 +0000 UTC m=+68.972235652"
Jul 20 15:51:44.967160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:44.967115    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:45.046142 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.046106    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:45.175861 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.175803    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:45.276527 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.276442    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:45.286088 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.285973    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t4wh7" event={"ID":"672c6793-5272-43d1-aa5d-ce5bf204f783","Type":"ContainerStarted","Data":"1ef922c7c56beb0b9190e0ee8e21ae59c47fa23787f4f3eff2cc00409d5b1dfa"}
Jul 20 15:51:45.287973 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.287911    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lplvs" event={"ID":"0892d712-0444-4ed9-8da9-41d802a56f8b","Type":"ContainerStarted","Data":"9fcc51f0d4be460df1427a546b12a3b3cfdb49a7b84d713c3e2f5cd42af10b52"}
Jul 20 15:51:45.291441 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.291101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6" event={"ID":"72f65cba-b2e7-4d1b-b943-c879e3175d9c","Type":"ContainerStarted","Data":"e70214a02ebaeaf80cf827c46ef6768eadb6fcc52e4fe5e23631408d9f3607b3"}
Jul 20 15:51:45.291441 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.291162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6" event={"ID":"72f65cba-b2e7-4d1b-b943-c879e3175d9c","Type":"ContainerStarted","Data":"2608e2e239770ead01792e58550d53a45de69ca8f4b6a45b552df4b48e29bf2f"}
Jul 20 15:51:45.291441 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.291186    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6" event={"ID":"72f65cba-b2e7-4d1b-b943-c879e3175d9c","Type":"ContainerStarted","Data":"dfddcc88a50e558d9cf8ca55b1adb574df7be68426553c55e1bbc9b24d23a222"}
Jul 20 15:51:45.295285 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.293673    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hggvm" event={"ID":"b63c3b63-90d2-45eb-ae0d-8491fa36761c","Type":"ContainerStarted","Data":"e8844e34b15fd3d209a0f2c2ff96cecf9c0ab2847436c173d27447b12dd93fab"}
Jul 20 15:51:45.296096 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.296076    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbfb6d5c-pjvz2"
Jul 20 15:51:45.330298 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.330182    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86475b7db4-tnph6" podStartSLOduration=2.596802555 podStartE2EDuration="10.33016065s" totalImagesPullingTime="7.733358095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:45.313335184 +0000 UTC m=+69.968598468" watchObservedRunningTime="2026-07-20 15:51:45.33016065 +0000 UTC m=+69.985423944"
Jul 20 15:51:45.335023 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.334950    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:45.466384 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.466310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:45.566203 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.566126    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:45.737651 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.737611    2564 patch_prober.go:28] interesting pod/image-registry-74dd6c6dcb-fxx4k container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 20 15:51:45.737819 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.737675    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k" podUID="f2ace347-d1c8-4286-9541-4885e78bb361" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 15:51:45.856725 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.856646    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:45.957392 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:45.957304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:46.072455 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:46.072377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:46.143943 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:46.143845    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:46.301216 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:46.301177    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lplvs" event={"ID":"0892d712-0444-4ed9-8da9-41d802a56f8b","Type":"ContainerStarted","Data":"cad6a3912653f11b18dd5fa8dacadd82b7b6e0d9b909b8c0a2d66f0a31b6a155"}
Jul 20 15:51:46.301706 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:46.301244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lplvs" event={"ID":"0892d712-0444-4ed9-8da9-41d802a56f8b","Type":"ContainerStarted","Data":"0fcf71965a7fb547139a932585983dbf8ec0da1bb35fabc238f47b7555d20490"}
Jul 20 15:51:46.319934 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:46.319871    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lplvs" podStartSLOduration=69.817472932 podStartE2EDuration="1m11.319847376s" totalImagesPullingTime="1.502374444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:46.317788019 +0000 UTC m=+70.973051296" watchObservedRunningTime="2026-07-20 15:51:46.319847376 +0000 UTC m=+70.975110653"
Jul 20 15:51:46.346061 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:46.345975    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:46.456715 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:46.456602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:46.516889 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:46.516852    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:46.648118 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:46.648016    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:46.841146 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:46.841110    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:47.286407 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:47.286368    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:47.874897 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:47.874864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:49.331531 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:49.331502    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:49.443931 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:49.443896    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:50.318397 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:50.318355    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t4wh7" event={"ID":"672c6793-5272-43d1-aa5d-ce5bf204f783","Type":"ContainerStarted","Data":"597c600f5df0f2bea56baa6edb584db884b8afb157e88b0e8a2b1063c2f14659"}
Jul 20 15:51:50.319860 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:50.319835    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hggvm" event={"ID":"b63c3b63-90d2-45eb-ae0d-8491fa36761c","Type":"ContainerStarted","Data":"891687b41f8607332eac45f14de9485213a6babe85f22ecd1502aeae1f5a7f24"}
Jul 20 15:51:50.319970 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:50.319927    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:51:50.332719 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:50.332651    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t4wh7" podStartSLOduration=69.327927205 podStartE2EDuration="1m14.332634078s" totalImagesPullingTime="5.004706873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:50.332587085 +0000 UTC m=+74.987850361" watchObservedRunningTime="2026-07-20 15:51:50.332634078 +0000 UTC m=+74.987897354"
Jul 20 15:51:50.345937 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:50.345887    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hggvm" podStartSLOduration=69.372108814 podStartE2EDuration="1m14.34587155s" totalImagesPullingTime="4.973762736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:50.345762163 +0000 UTC m=+75.001025439" watchObservedRunningTime="2026-07-20 15:51:50.34587155 +0000 UTC m=+75.001134825"
Jul 20 15:51:51.199816 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:51.199775    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:55.215925 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.215886    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:55.313372 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.313335    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:55.450807 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.450766    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:55.549470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.549432    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cztdc"]
Jul 20 15:51:55.553289 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.553260    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.555553 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.555529    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 20 15:51:55.556051 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.556028    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 20 15:51:55.556170 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.556052    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 20 15:51:55.556170 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.555635    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 20 15:51:55.556548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.556531    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cxvjb\""
Jul 20 15:51:55.556754 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.556737    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 20 15:51:55.558581 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.558564    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 20 15:51:55.563491 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.563467    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:55.636304 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.636268    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:55.732361 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.732326    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:55.734182 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.734164    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:51:55.734262 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.734168    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-sys\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.734262 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.734239    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vvbz\" (UniqueName: \"kubernetes.io/projected/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-kube-api-access-4vvbz\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.734334 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.734268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-textfile\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.734334 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.734297    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-root\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.734334 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.734322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-metrics-client-ca\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.734439 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.734347    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.734439 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.734379    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-tls\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.734503 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.734434    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.734503 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.734467    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-wtmp\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.833759 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.833657    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:55.835678 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.835657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-textfile\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.835744 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.835716    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-root\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.835781 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.835743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-metrics-client-ca\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.835781 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.835772    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.835876 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.835813    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-tls\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.835876 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.835820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-root\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.835876 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.835848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.836014 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.835878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-wtmp\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.836014 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.835914    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-sys\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.836014 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.835944    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vvbz\" (UniqueName: \"kubernetes.io/projected/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-kube-api-access-4vvbz\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.836161 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.836022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-sys\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.836161 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.836090    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-textfile\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.836261 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.836203    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-wtmp\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.836420 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.836397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.836482 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.836404    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-metrics-client-ca\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.838301 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.838271    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-tls\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.838418 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.838282    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.844943 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.844922    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vvbz\" (UniqueName: \"kubernetes.io/projected/2cd283b7-ea37-49fa-9ab6-adbc9909c8f0-kube-api-access-4vvbz\") pod \"node-exporter-cztdc\" (UID: \"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0\") " pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.865922 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.865892    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cztdc"
Jul 20 15:51:55.877424 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:51:55.877379    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cd283b7_ea37_49fa_9ab6_adbc9909c8f0.slice/crio-115e2e60ef22b848c6c8062eeed4f99608da035e21729f291685b463c3365515 WatchSource:0}: Error finding container 115e2e60ef22b848c6c8062eeed4f99608da035e21729f291685b463c3365515: Status 404 returned error can't find the container with id 115e2e60ef22b848c6c8062eeed4f99608da035e21729f291685b463c3365515
Jul 20 15:51:55.934029 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:55.933987    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:56.033896 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:56.033853    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:56.134349 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:56.134258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:56.340301 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:56.340265    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cztdc" event={"ID":"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0","Type":"ContainerStarted","Data":"115e2e60ef22b848c6c8062eeed4f99608da035e21729f291685b463c3365515"}
Jul 20 15:51:56.439245 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:56.439022    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:56.565885 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:56.565851    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:56.989589 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:56.989564    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:57.218338 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:57.218230    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:57.317979 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:57.317944    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:57.345167 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:57.345132    2564 generic.go:356] "Generic (PLEG): container finished" podID="2cd283b7-ea37-49fa-9ab6-adbc9909c8f0" containerID="3803157879e42f0e3896accd7eb6b4c7bf2b69e6f12b8cbc964023dd61e09945" exitCode=0
Jul 20 15:51:57.345555 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:57.345178    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cztdc" event={"ID":"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0","Type":"ContainerDied","Data":"3803157879e42f0e3896accd7eb6b4c7bf2b69e6f12b8cbc964023dd61e09945"}
Jul 20 15:51:58.351287 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:58.351156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cztdc" event={"ID":"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0","Type":"ContainerStarted","Data":"202bf5dcf074767eb192847f20a37e0290189627f2cb6ef15ad3086f15c99f4f"}
Jul 20 15:51:58.351287 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:58.351220    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cztdc" event={"ID":"2cd283b7-ea37-49fa-9ab6-adbc9909c8f0","Type":"ContainerStarted","Data":"faef8887f332d088ff7f9bbc10e87b63619239c6623df113f5b3d382e4cbb216"}
Jul 20 15:51:58.371506 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:58.371449    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cztdc" podStartSLOduration=2.255883317 podStartE2EDuration="3.371431531s" totalImagesPullingTime="1.050920648s" totalInitContainerRuntime="64.627566ms" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:58.370437548 +0000 UTC m=+83.025700823" watchObservedRunningTime="2026-07-20 15:51:58.371431531 +0000 UTC m=+83.026694807"
Jul 20 15:51:58.471550 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:58.471519    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:58.532522 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:58.532486    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:58.664601 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:58.664512    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:58.869136 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:58.869102    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:59.238845 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:59.238814    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69d78676d4-mds9q"
Jul 20 15:51:59.400919 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:59.400884    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:59.499012 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:59.498919    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:59.852953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:59.852916    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:59.953164 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:51:59.953111    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:00.231427 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:00.231348    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:00.292313 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:00.292277    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:00.715942 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:00.715913    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:00.752704 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:00.752651    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k" podUID="f2ace347-d1c8-4286-9541-4885e78bb361" containerName="registry" containerID="cri-o://79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077" gracePeriod=30
Jul 20 15:52:00.995587 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:00.995561    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:52:01.072196 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.072162    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:01.078882 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.078862    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-trusted-ca\") pod \"f2ace347-d1c8-4286-9541-4885e78bb361\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") "
Jul 20 15:52:01.078966 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.078905    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2ace347-d1c8-4286-9541-4885e78bb361-ca-trust-extracted\") pod \"f2ace347-d1c8-4286-9541-4885e78bb361\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") "
Jul 20 15:52:01.078966 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.078935    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls\") pod \"f2ace347-d1c8-4286-9541-4885e78bb361\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") "
Jul 20 15:52:01.078966 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.078959    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-installation-pull-secrets\") pod \"f2ace347-d1c8-4286-9541-4885e78bb361\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") "
Jul 20 15:52:01.079101 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.078981    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh5gn\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-kube-api-access-dh5gn\") pod \"f2ace347-d1c8-4286-9541-4885e78bb361\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") "
Jul 20 15:52:01.079101 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.079003    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-image-registry-private-configuration\") pod \"f2ace347-d1c8-4286-9541-4885e78bb361\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") "
Jul 20 15:52:01.079101 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.079027    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-registry-certificates\") pod \"f2ace347-d1c8-4286-9541-4885e78bb361\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") "
Jul 20 15:52:01.079101 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.079060    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-bound-sa-token\") pod \"f2ace347-d1c8-4286-9541-4885e78bb361\" (UID: \"f2ace347-d1c8-4286-9541-4885e78bb361\") "
Jul 20 15:52:01.079327 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.079300    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f2ace347-d1c8-4286-9541-4885e78bb361" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:52:01.079712 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.079670    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f2ace347-d1c8-4286-9541-4885e78bb361" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:52:01.081912 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.081875    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f2ace347-d1c8-4286-9541-4885e78bb361" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:52:01.082030 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.081967    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f2ace347-d1c8-4286-9541-4885e78bb361" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:52:01.082030 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.081973    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-kube-api-access-dh5gn" (OuterVolumeSpecName: "kube-api-access-dh5gn") pod "f2ace347-d1c8-4286-9541-4885e78bb361" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361"). InnerVolumeSpecName "kube-api-access-dh5gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:52:01.082030 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.082019    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f2ace347-d1c8-4286-9541-4885e78bb361" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:52:01.082135 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.082056    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f2ace347-d1c8-4286-9541-4885e78bb361" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:52:01.087749 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.087723    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2ace347-d1c8-4286-9541-4885e78bb361-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f2ace347-d1c8-4286-9541-4885e78bb361" (UID: "f2ace347-d1c8-4286-9541-4885e78bb361"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:52:01.136529 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.136496    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:01.179617 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.179576    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-image-registry-private-configuration\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:52:01.179617 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.179609    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-registry-certificates\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:52:01.179617 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.179620    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-bound-sa-token\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:52:01.179617 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.179629    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2ace347-d1c8-4286-9541-4885e78bb361-trusted-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:52:01.179874 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.179639    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2ace347-d1c8-4286-9541-4885e78bb361-ca-trust-extracted\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:52:01.179874 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.179647    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-registry-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:52:01.179874 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.179655    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2ace347-d1c8-4286-9541-4885e78bb361-installation-pull-secrets\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:52:01.179874 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.179663    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh5gn\" (UniqueName: \"kubernetes.io/projected/f2ace347-d1c8-4286-9541-4885e78bb361-kube-api-access-dh5gn\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:52:01.363446 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.363411    2564 generic.go:356] "Generic (PLEG): container finished" podID="f2ace347-d1c8-4286-9541-4885e78bb361" containerID="79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077" exitCode=0
Jul 20 15:52:01.363644 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.363453    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"
Jul 20 15:52:01.363644 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.363500    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k" event={"ID":"f2ace347-d1c8-4286-9541-4885e78bb361","Type":"ContainerDied","Data":"79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077"}
Jul 20 15:52:01.363644 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.363555    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74dd6c6dcb-fxx4k" event={"ID":"f2ace347-d1c8-4286-9541-4885e78bb361","Type":"ContainerDied","Data":"0891d7ffd9e7469fd064ac0568655595a236151883301d9120794ddc65247d99"}
Jul 20 15:52:01.363644 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.363583    2564 scope.go:117] "RemoveContainer" containerID="79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077"
Jul 20 15:52:01.377100 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.377076    2564 scope.go:117] "RemoveContainer" containerID="79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077"
Jul 20 15:52:01.377406 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:52:01.377384    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077\": container with ID starting with 79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077 not found: ID does not exist" containerID="79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077"
Jul 20 15:52:01.377453 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.377417    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077"} err="failed to get container status \"79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077\": rpc error: code = NotFound desc = could not find container \"79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077\": container with ID starting with 79d0639ff168730cdb69797dbb0e2cd56d0042710f5eeac4e1b264380a3ba077 not found: ID does not exist"
Jul 20 15:52:01.387870 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.387843    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"]
Jul 20 15:52:01.393030 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.393009    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74dd6c6dcb-fxx4k"]
Jul 20 15:52:01.850261 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.850228    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:01.867505 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:01.867474    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2ace347-d1c8-4286-9541-4885e78bb361" path="/var/lib/kubelet/pods/f2ace347-d1c8-4286-9541-4885e78bb361/volumes"
Jul 20 15:52:02.882441 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:02.882400    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:02.982302 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:02.982271    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:03.182641 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:03.182552    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:03.242548 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:03.242502    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:03.758382 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:03.758349    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:03.858103 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:03.858030    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:03.963840 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:03.963809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:04.502225 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:04.502192    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:04.709988 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:04.709959    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:04.779850 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:04.779819    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:05.665894 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:05.665859    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:05.723766 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:05.723734    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:09.133148 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:09.133114    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:09.244252 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:09.244210    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:11.914953 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:11.914917    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:12.015890 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:12.015855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:12.229093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:12.229010    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:12.329106 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:12.329079    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:13.297234 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:13.297201    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:13.386211 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:13.386176    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:21.325756 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:21.325716    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hggvm"
Jul 20 15:52:26.226002 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:26.225967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:26.321609 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:26.321578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:26.410326 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:26.410291    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:27.249121 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:27.249087    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:27.357067 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:27.357031    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:27.456727 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:27.456681    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:27.560449 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:52:27.560420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:08.532188 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:08.532154    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:11.634589 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:11.634555    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:11.684574 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:11.684540    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:23.538436 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:23.538376    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:23.608607 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:23.608576    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:27.219873 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:27.219837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:27.322263 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:27.322234    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:27.422748 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:27.422721    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:31.589907 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:31.589827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:31.644896 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:31.644861    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:45.056474 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:45.056430    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:45.113021 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:45.112988    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:45.169260 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:45.169227    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:45.968375 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:45.968343    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:46.025342 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:46.025305    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:55.669400 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:55.669365    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:55.727904 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:55.727874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:55.779780 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:55.779751    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:56.344425 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:56.344390    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:56.396137 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:53:56.396087    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:14.342527 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:14.342494    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:14.436298 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:14.436266    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:33.693181 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:33.693146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:36.085584 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:36.085515    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:37.479070 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:37.479030    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:39.712198 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:39.712168    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:39.814978 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:39.814947    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:41.207847 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:41.207813    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:41.361890 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:41.361829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:41.624293 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:41.624262    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:41.714237 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:41.714211    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:44.009208 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:44.009171    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:44.242872 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:44.242835    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:46.738943 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:46.738908    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:46.822069 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:46.822039    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:51.695675 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:51.695636    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:54.333656 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:54.333622    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:54.424948 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:54.424914    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:55.870373 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:55.870344    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:59.521849 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:59.521756    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:59.620269 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:54:59.620236    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:01.435354 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:01.435323    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:03.238948 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:03.238915    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:03.352849 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:03.352822    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:18.430084 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:18.430044    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:18.527193 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:18.527161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:19.009483 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.009451    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:19.340557 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.340523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:19.439366 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.439333    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:19.537087 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.537053    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"]
Jul 20 15:55:19.537300 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.537288    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2ace347-d1c8-4286-9541-4885e78bb361" containerName="registry"
Jul 20 15:55:19.537345 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.537301    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2ace347-d1c8-4286-9541-4885e78bb361" containerName="registry"
Jul 20 15:55:19.537378 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.537345    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2ace347-d1c8-4286-9541-4885e78bb361" containerName="registry"
Jul 20 15:55:19.540106 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.540090    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.542338 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.542316    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 20 15:55:19.542338 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.542328    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 20 15:55:19.542530 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.542407    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 20 15:55:19.542655 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.542640    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 20 15:55:19.544456 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.544441    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ft7hk\""
Jul 20 15:55:19.552759 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.552736    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"]
Jul 20 15:55:19.702059 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.701965    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/623fe211-2e84-4558-bbda-856abfa7a705-manifests\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.702059 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.702014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/623fe211-2e84-4558-bbda-856abfa7a705-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.702251 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.702110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/623fe211-2e84-4558-bbda-856abfa7a705-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.702251 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.702146    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/623fe211-2e84-4558-bbda-856abfa7a705-tmp\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.702251 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.702175    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79xhh\" (UniqueName: \"kubernetes.io/projected/623fe211-2e84-4558-bbda-856abfa7a705-kube-api-access-79xhh\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.755630 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.755602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:19.802764 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.802717    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/623fe211-2e84-4558-bbda-856abfa7a705-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.802960 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.802775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/623fe211-2e84-4558-bbda-856abfa7a705-tmp\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.802960 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.802798    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79xhh\" (UniqueName: \"kubernetes.io/projected/623fe211-2e84-4558-bbda-856abfa7a705-kube-api-access-79xhh\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.802960 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.802832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/623fe211-2e84-4558-bbda-856abfa7a705-manifests\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.802960 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.802860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/623fe211-2e84-4558-bbda-856abfa7a705-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.803281 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.803258    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/623fe211-2e84-4558-bbda-856abfa7a705-manifests\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.805057 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.805032    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/623fe211-2e84-4558-bbda-856abfa7a705-tmp\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.805401 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.805378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/623fe211-2e84-4558-bbda-856abfa7a705-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.805443 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.805378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/623fe211-2e84-4558-bbda-856abfa7a705-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.810857 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.810833    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79xhh\" (UniqueName: \"kubernetes.io/projected/623fe211-2e84-4558-bbda-856abfa7a705-kube-api-access-79xhh\") pod \"opendatahub-operator-controller-manager-5b76897bc6-f8cng\" (UID: \"623fe211-2e84-4558-bbda-856abfa7a705\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.850447 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.850408    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:19.990983 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:19.990904    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"]
Jul 20 15:55:19.994143 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:55:19.994113    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623fe211_2e84_4558_bbda_856abfa7a705.slice/crio-af9ebb343a78c434af1a8ebf4b1631946e304c74b55b6eed2b08aa1ba56a10a8 WatchSource:0}: Error finding container af9ebb343a78c434af1a8ebf4b1631946e304c74b55b6eed2b08aa1ba56a10a8: Status 404 returned error can't find the container with id af9ebb343a78c434af1a8ebf4b1631946e304c74b55b6eed2b08aa1ba56a10a8
Jul 20 15:55:20.021485 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:20.021445    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng" event={"ID":"623fe211-2e84-4558-bbda-856abfa7a705","Type":"ContainerStarted","Data":"af9ebb343a78c434af1a8ebf4b1631946e304c74b55b6eed2b08aa1ba56a10a8"}
Jul 20 15:55:20.129007 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:20.128973    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:20.230312 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:20.230272    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:21.808189 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:21.808155    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:21.913103 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:21.913072    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:23.034858 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:23.034823    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng" event={"ID":"623fe211-2e84-4558-bbda-856abfa7a705","Type":"ContainerStarted","Data":"c7a0b7ece37361a176e98379e865e25fc35ab9f6134e20993ae6c35c5bd07664"}
Jul 20 15:55:24.040005 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:24.039971    2564 generic.go:356] "Generic (PLEG): container finished" podID="623fe211-2e84-4558-bbda-856abfa7a705" containerID="c7a0b7ece37361a176e98379e865e25fc35ab9f6134e20993ae6c35c5bd07664" exitCode=0
Jul 20 15:55:24.040455 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:24.040040    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng" event={"ID":"623fe211-2e84-4558-bbda-856abfa7a705","Type":"ContainerDied","Data":"c7a0b7ece37361a176e98379e865e25fc35ab9f6134e20993ae6c35c5bd07664"}
Jul 20 15:55:25.046460 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:25.046420    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng" event={"ID":"623fe211-2e84-4558-bbda-856abfa7a705","Type":"ContainerStarted","Data":"d50d2e378454c27245f46daff1ce4f7b3e173bd5d73962523b46c0b0af5fc008"}
Jul 20 15:55:25.046863 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:25.046535    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:25.071488 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:25.071431    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng" podStartSLOduration=1.817210116 podStartE2EDuration="6.071413659s" totalImagesPullingTime="3.199766029s" totalInitContainerRuntime="1.054437514s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:55:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:55:25.069833938 +0000 UTC m=+289.725097259" watchObservedRunningTime="2026-07-20 15:55:25.071413659 +0000 UTC m=+289.726676934"
Jul 20 15:55:31.187942 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:31.187912    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:32.601904 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:32.601864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:35.763381 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:35.763346    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 15:55:35.769412 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:35.769387    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 20 15:55:35.770215 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:35.770192    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 15:55:36.052173 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:36.052147    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-f8cng"
Jul 20 15:55:38.843308 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:38.843275    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:38.962351 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:38.962314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:39.831664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:39.831631    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:39.941643 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:39.941608    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:40.100206 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:40.100115    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:40.212869 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:40.212829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:40.796390 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:40.796362    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:41.120430 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:41.120334    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:47.831670 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:47.831636    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:50.053292 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.053255    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:50.178031 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.177994    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"]
Jul 20 15:55:50.182326 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.182302    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:50.187175 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.187147    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 20 15:55:50.187296 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.187153    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 20 15:55:50.187445 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.187429    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-m84xq\""
Jul 20 15:55:50.189082 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.189059    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 20 15:55:50.191852 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.191836    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 20 15:55:50.206552 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.206517    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"]
Jul 20 15:55:50.224862 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.224822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f0545fb-6ba1-45c6-8fac-d6717a671111-tmp\") pod \"kube-auth-proxy-bc6c54cc6-rb9cf\" (UID: \"8f0545fb-6ba1-45c6-8fac-d6717a671111\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:50.225041 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.224884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0545fb-6ba1-45c6-8fac-d6717a671111-tls-certs\") pod \"kube-auth-proxy-bc6c54cc6-rb9cf\" (UID: \"8f0545fb-6ba1-45c6-8fac-d6717a671111\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:50.225041 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.224914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7c5d\" (UniqueName: \"kubernetes.io/projected/8f0545fb-6ba1-45c6-8fac-d6717a671111-kube-api-access-n7c5d\") pod \"kube-auth-proxy-bc6c54cc6-rb9cf\" (UID: \"8f0545fb-6ba1-45c6-8fac-d6717a671111\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:50.325514 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.325412    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0545fb-6ba1-45c6-8fac-d6717a671111-tls-certs\") pod \"kube-auth-proxy-bc6c54cc6-rb9cf\" (UID: \"8f0545fb-6ba1-45c6-8fac-d6717a671111\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:50.325514 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.325469    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7c5d\" (UniqueName: \"kubernetes.io/projected/8f0545fb-6ba1-45c6-8fac-d6717a671111-kube-api-access-n7c5d\") pod \"kube-auth-proxy-bc6c54cc6-rb9cf\" (UID: \"8f0545fb-6ba1-45c6-8fac-d6717a671111\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:50.325757 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.325541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f0545fb-6ba1-45c6-8fac-d6717a671111-tmp\") pod \"kube-auth-proxy-bc6c54cc6-rb9cf\" (UID: \"8f0545fb-6ba1-45c6-8fac-d6717a671111\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:50.325757 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:55:50.325546    2564 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 20 15:55:50.325757 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:55:50.325623    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0545fb-6ba1-45c6-8fac-d6717a671111-tls-certs podName:8f0545fb-6ba1-45c6-8fac-d6717a671111 nodeName:}" failed. No retries permitted until 2026-07-20 15:55:50.825601802 +0000 UTC m=+315.480865075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8f0545fb-6ba1-45c6-8fac-d6717a671111-tls-certs") pod "kube-auth-proxy-bc6c54cc6-rb9cf" (UID: "8f0545fb-6ba1-45c6-8fac-d6717a671111") : secret "kube-auth-proxy-tls" not found
Jul 20 15:55:50.327765 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.327747    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f0545fb-6ba1-45c6-8fac-d6717a671111-tmp\") pod \"kube-auth-proxy-bc6c54cc6-rb9cf\" (UID: \"8f0545fb-6ba1-45c6-8fac-d6717a671111\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:50.337238 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.337204    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7c5d\" (UniqueName: \"kubernetes.io/projected/8f0545fb-6ba1-45c6-8fac-d6717a671111-kube-api-access-n7c5d\") pod \"kube-auth-proxy-bc6c54cc6-rb9cf\" (UID: \"8f0545fb-6ba1-45c6-8fac-d6717a671111\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:50.530234 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.530199    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:50.829563 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.829520    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0545fb-6ba1-45c6-8fac-d6717a671111-tls-certs\") pod \"kube-auth-proxy-bc6c54cc6-rb9cf\" (UID: \"8f0545fb-6ba1-45c6-8fac-d6717a671111\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:50.831992 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:50.831971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0545fb-6ba1-45c6-8fac-d6717a671111-tls-certs\") pod \"kube-auth-proxy-bc6c54cc6-rb9cf\" (UID: \"8f0545fb-6ba1-45c6-8fac-d6717a671111\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:51.091642 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.091550    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"
Jul 20 15:55:51.232217 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.232192    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf"]
Jul 20 15:55:51.234793 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:55:51.234766    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0545fb_6ba1_45c6_8fac_d6717a671111.slice/crio-e3f75740e6d99931c6ace0406000919962187b1262215798d5fdb493f70c39b1 WatchSource:0}: Error finding container e3f75740e6d99931c6ace0406000919962187b1262215798d5fdb493f70c39b1: Status 404 returned error can't find the container with id e3f75740e6d99931c6ace0406000919962187b1262215798d5fdb493f70c39b1
Jul 20 15:55:51.236520 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.236505    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 15:55:51.586972 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.586928    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:51.740902 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.740867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:51.782620 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.782585    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"]
Jul 20 15:55:51.786806 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.786787    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:55:51.789737 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.789712    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xsdtp\""
Jul 20 15:55:51.789859 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.789839    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 20 15:55:51.827985 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.827950    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"]
Jul 20 15:55:51.836155 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.836121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww4f9\" (UniqueName: \"kubernetes.io/projected/589ce640-c965-4058-87c3-e2e3eb813352-kube-api-access-ww4f9\") pod \"kserve-controller-manager-d6f4b65b8-9xmww\" (UID: \"589ce640-c965-4058-87c3-e2e3eb813352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:55:51.836336 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.836173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/589ce640-c965-4058-87c3-e2e3eb813352-cert\") pod \"kserve-controller-manager-d6f4b65b8-9xmww\" (UID: \"589ce640-c965-4058-87c3-e2e3eb813352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:55:51.836477 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.836460    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"]
Jul 20 15:55:51.839604 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.839556    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"
Jul 20 15:55:51.842999 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.842976    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pj6dx\""
Jul 20 15:55:51.843152 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.843132    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 20 15:55:51.863056 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.863017    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"]
Jul 20 15:55:51.903170 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.903123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:51.936947 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.936913    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/523b88b3-dd29-495c-b568-d04767ffe45a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pr94l\" (UID: \"523b88b3-dd29-495c-b568-d04767ffe45a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"
Jul 20 15:55:51.937183 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.936990    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ch5f\" (UniqueName: \"kubernetes.io/projected/523b88b3-dd29-495c-b568-d04767ffe45a-kube-api-access-7ch5f\") pod \"llmisvc-controller-manager-777c6bd7d5-pr94l\" (UID: \"523b88b3-dd29-495c-b568-d04767ffe45a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"
Jul 20 15:55:51.937183 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.937061    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww4f9\" (UniqueName: \"kubernetes.io/projected/589ce640-c965-4058-87c3-e2e3eb813352-kube-api-access-ww4f9\") pod \"kserve-controller-manager-d6f4b65b8-9xmww\" (UID: \"589ce640-c965-4058-87c3-e2e3eb813352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:55:51.937183 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.937102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/589ce640-c965-4058-87c3-e2e3eb813352-cert\") pod \"kserve-controller-manager-d6f4b65b8-9xmww\" (UID: \"589ce640-c965-4058-87c3-e2e3eb813352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:55:51.937316 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:55:51.937199    2564 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 20 15:55:51.937316 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:55:51.937266    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589ce640-c965-4058-87c3-e2e3eb813352-cert podName:589ce640-c965-4058-87c3-e2e3eb813352 nodeName:}" failed. No retries permitted until 2026-07-20 15:55:52.437242759 +0000 UTC m=+317.092506040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/589ce640-c965-4058-87c3-e2e3eb813352-cert") pod "kserve-controller-manager-d6f4b65b8-9xmww" (UID: "589ce640-c965-4058-87c3-e2e3eb813352") : secret "kserve-webhook-server-cert" not found
Jul 20 15:55:51.964052 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:51.964016    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww4f9\" (UniqueName: \"kubernetes.io/projected/589ce640-c965-4058-87c3-e2e3eb813352-kube-api-access-ww4f9\") pod \"kserve-controller-manager-d6f4b65b8-9xmww\" (UID: \"589ce640-c965-4058-87c3-e2e3eb813352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:55:52.002509 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.002467    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:52.037812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.037771    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/523b88b3-dd29-495c-b568-d04767ffe45a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pr94l\" (UID: \"523b88b3-dd29-495c-b568-d04767ffe45a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"
Jul 20 15:55:52.037991 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.037851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ch5f\" (UniqueName: \"kubernetes.io/projected/523b88b3-dd29-495c-b568-d04767ffe45a-kube-api-access-7ch5f\") pod \"llmisvc-controller-manager-777c6bd7d5-pr94l\" (UID: \"523b88b3-dd29-495c-b568-d04767ffe45a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"
Jul 20 15:55:52.040783 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.040752    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/523b88b3-dd29-495c-b568-d04767ffe45a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pr94l\" (UID: \"523b88b3-dd29-495c-b568-d04767ffe45a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"
Jul 20 15:55:52.049132 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.049094    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ch5f\" (UniqueName: \"kubernetes.io/projected/523b88b3-dd29-495c-b568-d04767ffe45a-kube-api-access-7ch5f\") pod \"llmisvc-controller-manager-777c6bd7d5-pr94l\" (UID: \"523b88b3-dd29-495c-b568-d04767ffe45a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"
Jul 20 15:55:52.142368 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.142288    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf" event={"ID":"8f0545fb-6ba1-45c6-8fac-d6717a671111","Type":"ContainerStarted","Data":"e3f75740e6d99931c6ace0406000919962187b1262215798d5fdb493f70c39b1"}
Jul 20 15:55:52.152566 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.152533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"
Jul 20 15:55:52.356511 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.356478    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"]
Jul 20 15:55:52.359677 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:55:52.359651    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod523b88b3_dd29_495c_b568_d04767ffe45a.slice/crio-d207ce0c9c93073795b931e51b351c62a34962d0f6e6d6699fb4d4ecfb2c32b0 WatchSource:0}: Error finding container d207ce0c9c93073795b931e51b351c62a34962d0f6e6d6699fb4d4ecfb2c32b0: Status 404 returned error can't find the container with id d207ce0c9c93073795b931e51b351c62a34962d0f6e6d6699fb4d4ecfb2c32b0
Jul 20 15:55:52.440640 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.440544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/589ce640-c965-4058-87c3-e2e3eb813352-cert\") pod \"kserve-controller-manager-d6f4b65b8-9xmww\" (UID: \"589ce640-c965-4058-87c3-e2e3eb813352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:55:52.443183 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.442026    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:52.443864 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.443839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/589ce640-c965-4058-87c3-e2e3eb813352-cert\") pod \"kserve-controller-manager-d6f4b65b8-9xmww\" (UID: \"589ce640-c965-4058-87c3-e2e3eb813352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:55:52.696892 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.696796    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:55:52.853107 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.853076    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"]
Jul 20 15:55:52.857007 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:55:52.856966    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589ce640_c965_4058_87c3_e2e3eb813352.slice/crio-496a3940cb7933f7bcf4d8f175c2d688d971fdd85488dfef2b01bc70b5814998 WatchSource:0}: Error finding container 496a3940cb7933f7bcf4d8f175c2d688d971fdd85488dfef2b01bc70b5814998: Status 404 returned error can't find the container with id 496a3940cb7933f7bcf4d8f175c2d688d971fdd85488dfef2b01bc70b5814998
Jul 20 15:55:52.976124 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:52.976018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:53.059423 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.059385    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"]
Jul 20 15:55:53.062513 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.062487    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.069443 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.069401    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 20 15:55:53.069613 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.069445    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 20 15:55:53.069613 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.069597    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 20 15:55:53.069884 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.069637    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:53.069925 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.069887    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 15:55:53.079905 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.079879    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xjhmb\""
Jul 20 15:55:53.087344 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.087316    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"]
Jul 20 15:55:53.145500 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.145467    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.145955 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.145507    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.145955 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.145567    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a0c8fb0-d84c-432c-96f3-55f0e5312188-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.145955 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.145610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.145955 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.145730    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wb27\" (UniqueName: \"kubernetes.io/projected/1a0c8fb0-d84c-432c-96f3-55f0e5312188-kube-api-access-5wb27\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.145955 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.145774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1a0c8fb0-d84c-432c-96f3-55f0e5312188-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.145955 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.145872    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.147375 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.147336    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww" event={"ID":"589ce640-c965-4058-87c3-e2e3eb813352","Type":"ContainerStarted","Data":"496a3940cb7933f7bcf4d8f175c2d688d971fdd85488dfef2b01bc70b5814998"}
Jul 20 15:55:53.148584 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.148551    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l" event={"ID":"523b88b3-dd29-495c-b568-d04767ffe45a","Type":"ContainerStarted","Data":"d207ce0c9c93073795b931e51b351c62a34962d0f6e6d6699fb4d4ecfb2c32b0"}
Jul 20 15:55:53.164843 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.164813    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:53.246465 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.246378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wb27\" (UniqueName: \"kubernetes.io/projected/1a0c8fb0-d84c-432c-96f3-55f0e5312188-kube-api-access-5wb27\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.246465 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.246434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1a0c8fb0-d84c-432c-96f3-55f0e5312188-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.246659 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.246481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.246659 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.246529    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.246659 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.246552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.246659 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.246574    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a0c8fb0-d84c-432c-96f3-55f0e5312188-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.246659 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.246599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.247374 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.247329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.248045 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.248015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:53.249327 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.249296    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a0c8fb0-d84c-432c-96f3-55f0e5312188-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.249422 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.249394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1a0c8fb0-d84c-432c-96f3-55f0e5312188-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.249648 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.249631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.249967 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.249948    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.255676 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.255643    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wb27\" (UniqueName: \"kubernetes.io/projected/1a0c8fb0-d84c-432c-96f3-55f0e5312188-kube-api-access-5wb27\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.256437 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.256410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a0c8fb0-d84c-432c-96f3-55f0e5312188-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hpbz4\" (UID: \"1a0c8fb0-d84c-432c-96f3-55f0e5312188\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:53.307612 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.307577    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:53.374926 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:53.374888    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:54.048032 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:54.047987    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"]
Jul 20 15:55:54.054882 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:55:54.054848    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0c8fb0_d84c_432c_96f3_55f0e5312188.slice/crio-90adf688954db7aebd98002370c40d183f0058c1ac87bfe975b28a6195c6f2fa WatchSource:0}: Error finding container 90adf688954db7aebd98002370c40d183f0058c1ac87bfe975b28a6195c6f2fa: Status 404 returned error can't find the container with id 90adf688954db7aebd98002370c40d183f0058c1ac87bfe975b28a6195c6f2fa
Jul 20 15:55:54.155276 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:54.155182    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf" event={"ID":"8f0545fb-6ba1-45c6-8fac-d6717a671111","Type":"ContainerStarted","Data":"3e064ac9b7ff21ea4dff46700cd23e7f1ef7d16bf5c846e6e304b833e06dcbc3"}
Jul 20 15:55:54.157152 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:54.157121    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4" event={"ID":"1a0c8fb0-d84c-432c-96f3-55f0e5312188","Type":"ContainerStarted","Data":"90adf688954db7aebd98002370c40d183f0058c1ac87bfe975b28a6195c6f2fa"}
Jul 20 15:55:54.174627 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:54.174327    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-rb9cf" podStartSLOduration=1.356157563 podStartE2EDuration="4.174306916s" totalImagesPullingTime="2.818149353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:55:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:55:54.173152604 +0000 UTC m=+318.828415881" watchObservedRunningTime="2026-07-20 15:55:54.174306916 +0000 UTC m=+318.829570192"
Jul 20 15:55:54.841375 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:54.841317    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:57.572779 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:57.572738    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 15:55:57.573072 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:57.572811    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 15:55:58.175792 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.175756    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4" event={"ID":"1a0c8fb0-d84c-432c-96f3-55f0e5312188","Type":"ContainerStarted","Data":"33588bc6a70f82621e6d94a27d29d3e7cbe17c696e19cd042542480c9c11e5f1"}
Jul 20 15:55:58.175998 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.175934    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:55:58.177718 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.177671    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-hpbz4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 15:55:58.177863 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.177745    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4" podUID="1a0c8fb0-d84c-432c-96f3-55f0e5312188" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 15:55:58.178017 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.177988    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l" event={"ID":"523b88b3-dd29-495c-b568-d04767ffe45a","Type":"ContainerStarted","Data":"0a51ddd6cc1c38e92a3c44e384a7be23e39d42d08243721e45996a1110a17603"}
Jul 20 15:55:58.178146 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.178104    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"
Jul 20 15:55:58.185210 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.185170    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww" event={"ID":"589ce640-c965-4058-87c3-e2e3eb813352","Type":"ContainerStarted","Data":"fa9735fb4e2f2675a65beddc91114728bcd6cd8dbe2737d093e6fde8f4bad5fd"}
Jul 20 15:55:58.185364 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.185298    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:55:58.215921 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.215869    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4" podStartSLOduration=2.7004910669999997 podStartE2EDuration="6.215852341s" totalImagesPullingTime="3.515361274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:55:58.214769107 +0000 UTC m=+322.870032384" watchObservedRunningTime="2026-07-20 15:55:58.215852341 +0000 UTC m=+322.871115595"
Jul 20 15:55:58.234703 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.234634    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l" podStartSLOduration=2.021009668 podStartE2EDuration="7.234617402s" totalImagesPullingTime="5.213607734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:55:58.232859642 +0000 UTC m=+322.888122918" watchObservedRunningTime="2026-07-20 15:55:58.234617402 +0000 UTC m=+322.889880678"
Jul 20 15:55:58.255305 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.255254    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww" podStartSLOduration=2.544824371 podStartE2EDuration="7.255233349s" totalImagesPullingTime="4.710408978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:55:58.255036158 +0000 UTC m=+322.910299424" watchObservedRunningTime="2026-07-20 15:55:58.255233349 +0000 UTC m=+322.910496626"
Jul 20 15:55:58.404246 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.404210    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:58.697043 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:58.697008    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:59.189995 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:59.189964    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-hpbz4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 15:55:59.190160 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:59.190017    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4" podUID="1a0c8fb0-d84c-432c-96f3-55f0e5312188" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 15:55:59.415831 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:59.415789    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:59.556178 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:59.556127    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:59.744499 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:55:59.744460    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:02.190117 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:56:02.190085    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hpbz4"
Jul 20 15:56:29.191554 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:56:29.191477    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pr94l"
Jul 20 15:56:29.194477 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:56:29.194458    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9xmww"
Jul 20 15:56:40.427095 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:56:40.427008    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:40.519067 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:56:40.519026    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:40.657652 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:56:40.657618    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:41.370346 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:56:41.370298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:41.585241 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:56:41.585210    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:09.484445 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:09.484412    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:12.958663 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:12.958627    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:14.928938 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:14.928899    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:16.290292 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.290253    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"]
Jul 20 15:57:16.292530 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.292512    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:16.294809 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.294783    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 20 15:57:16.295353 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.295328    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pfz48\""
Jul 20 15:57:16.295353 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.295342    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 20 15:57:16.308817 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.308793    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"]
Jul 20 15:57:16.394067 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.394027    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh2gg\" (UniqueName: \"kubernetes.io/projected/d1af4d36-762f-4202-a3da-c30f67ea44a4-kube-api-access-rh2gg\") pod \"limitador-operator-controller-manager-859d7b78bd-thnxx\" (UID: \"d1af4d36-762f-4202-a3da-c30f67ea44a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:16.494638 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.494598    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh2gg\" (UniqueName: \"kubernetes.io/projected/d1af4d36-762f-4202-a3da-c30f67ea44a4-kube-api-access-rh2gg\") pod \"limitador-operator-controller-manager-859d7b78bd-thnxx\" (UID: \"d1af4d36-762f-4202-a3da-c30f67ea44a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:16.498261 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.498241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:16.504560 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.504531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh2gg\" (UniqueName: \"kubernetes.io/projected/d1af4d36-762f-4202-a3da-c30f67ea44a4-kube-api-access-rh2gg\") pod \"limitador-operator-controller-manager-859d7b78bd-thnxx\" (UID: \"d1af4d36-762f-4202-a3da-c30f67ea44a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:16.602614 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.602509    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:16.730864 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.730834    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"]
Jul 20 15:57:16.733939 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:57:16.733902    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1af4d36_762f_4202_a3da_c30f67ea44a4.slice/crio-33446a467d84013b765d629bfc837a0e7ca916e5b1187e59b99f1811c8b4b99c WatchSource:0}: Error finding container 33446a467d84013b765d629bfc837a0e7ca916e5b1187e59b99f1811c8b4b99c: Status 404 returned error can't find the container with id 33446a467d84013b765d629bfc837a0e7ca916e5b1187e59b99f1811c8b4b99c
Jul 20 15:57:16.871802 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:16.871725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:17.469581 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:17.469541    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx" event={"ID":"d1af4d36-762f-4202-a3da-c30f67ea44a4","Type":"ContainerStarted","Data":"33446a467d84013b765d629bfc837a0e7ca916e5b1187e59b99f1811c8b4b99c"}
Jul 20 15:57:18.191859 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:18.191826    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:19.482625 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:19.482590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx" event={"ID":"d1af4d36-762f-4202-a3da-c30f67ea44a4","Type":"ContainerStarted","Data":"6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604"}
Jul 20 15:57:19.483073 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:19.482725    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:19.501193 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:19.501139    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx" podStartSLOduration=1.678714336 podStartE2EDuration="3.50112306s" totalImagesPullingTime="1.822408724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:57:19.499925627 +0000 UTC m=+404.155188904" watchObservedRunningTime="2026-07-20 15:57:19.50112306 +0000 UTC m=+404.156386376"
Jul 20 15:57:20.041449 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:20.041414    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:20.146143 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:20.146112    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:20.292885 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:20.292795    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:20.542384 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:20.542352    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:20.650819 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:20.650734    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:22.583480 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:22.583450    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:23.480617 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:23.480582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:23.644968 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:23.644936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:25.242569 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:25.242537    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:25.456827 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:25.456782    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:30.489406 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:30.489374    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:30.895166 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:30.895132    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:30.996470 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:30.996431    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:33.466021 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:33.465990    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:33.580772 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:33.580740    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:34.022634 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.022594    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"]
Jul 20 15:57:34.022901 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.022870    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" containerName="manager" containerID="cri-o://6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604" gracePeriod=2
Jul 20 15:57:34.027746 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.027715    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"]
Jul 20 15:57:34.070137 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.070099    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4"]
Jul 20 15:57:34.070379 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.070367    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" containerName="manager"
Jul 20 15:57:34.070442 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.070380    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" containerName="manager"
Jul 20 15:57:34.070442 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.070432    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" containerName="manager"
Jul 20 15:57:34.073394 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.073373    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4"
Jul 20 15:57:34.093337 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.093010    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4"]
Jul 20 15:57:34.115450 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:57:34.115397    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-thnxx\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:34.137552 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.137512    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgxqg\" (UniqueName: \"kubernetes.io/projected/fd3b5025-3574-410f-8634-e0858fc5e985-kube-api-access-jgxqg\") pod \"limitador-operator-controller-manager-859d7b78bd-jspg4\" (UID: \"fd3b5025-3574-410f-8634-e0858fc5e985\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4"
Jul 20 15:57:34.238675 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.238637    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgxqg\" (UniqueName: \"kubernetes.io/projected/fd3b5025-3574-410f-8634-e0858fc5e985-kube-api-access-jgxqg\") pod \"limitador-operator-controller-manager-859d7b78bd-jspg4\" (UID: \"fd3b5025-3574-410f-8634-e0858fc5e985\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4"
Jul 20 15:57:34.263429 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.263395    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgxqg\" (UniqueName: \"kubernetes.io/projected/fd3b5025-3574-410f-8634-e0858fc5e985-kube-api-access-jgxqg\") pod \"limitador-operator-controller-manager-859d7b78bd-jspg4\" (UID: \"fd3b5025-3574-410f-8634-e0858fc5e985\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4"
Jul 20 15:57:34.266858 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.266833    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:34.269021 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:57:34.268984    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-thnxx\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:34.339533 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.339440    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh2gg\" (UniqueName: \"kubernetes.io/projected/d1af4d36-762f-4202-a3da-c30f67ea44a4-kube-api-access-rh2gg\") pod \"d1af4d36-762f-4202-a3da-c30f67ea44a4\" (UID: \"d1af4d36-762f-4202-a3da-c30f67ea44a4\") "
Jul 20 15:57:34.341576 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.341550    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1af4d36-762f-4202-a3da-c30f67ea44a4-kube-api-access-rh2gg" (OuterVolumeSpecName: "kube-api-access-rh2gg") pod "d1af4d36-762f-4202-a3da-c30f67ea44a4" (UID: "d1af4d36-762f-4202-a3da-c30f67ea44a4"). InnerVolumeSpecName "kube-api-access-rh2gg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:57:34.397467 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.397433    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4"
Jul 20 15:57:34.440746 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.440714    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh2gg\" (UniqueName: \"kubernetes.io/projected/d1af4d36-762f-4202-a3da-c30f67ea44a4-kube-api-access-rh2gg\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:57:34.545244 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.545207    2564 generic.go:356] "Generic (PLEG): container finished" podID="d1af4d36-762f-4202-a3da-c30f67ea44a4" containerID="6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604" exitCode=0
Jul 20 15:57:34.545670 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.545296    2564 scope.go:117] "RemoveContainer" containerID="6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604"
Jul 20 15:57:34.545670 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.545305    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:34.547926 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:57:34.547888    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-thnxx\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:34.553622 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.553602    2564 scope.go:117] "RemoveContainer" containerID="6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604"
Jul 20 15:57:34.553914 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:57:34.553896    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604\": container with ID starting with 6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604 not found: ID does not exist" containerID="6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604"
Jul 20 15:57:34.553987 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.553925    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604"} err="failed to get container status \"6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604\": rpc error: code = NotFound desc = could not find container \"6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604\": container with ID starting with 6c1f64acd07cf659b85a86bab122744627a16e6f133b293091677f9ac6f25604 not found: ID does not exist"
Jul 20 15:57:34.556584 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:57:34.556556    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-thnxx\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:34.559343 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:34.559317    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4"]
Jul 20 15:57:34.560084 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:57:34.560060    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3b5025_3574_410f_8634_e0858fc5e985.slice/crio-f9eae8a0dcff48a00c319abb0574ed7bf9422a41062b216b0b5637ee264544d6 WatchSource:0}: Error finding container f9eae8a0dcff48a00c319abb0574ed7bf9422a41062b216b0b5637ee264544d6: Status 404 returned error can't find the container with id f9eae8a0dcff48a00c319abb0574ed7bf9422a41062b216b0b5637ee264544d6
Jul 20 15:57:35.550737 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:35.550683    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4" event={"ID":"fd3b5025-3574-410f-8634-e0858fc5e985","Type":"ContainerStarted","Data":"e52760c9456ea4aa80269616c75df9e2174f9e301cf7c5e06f3db347b0025357"}
Jul 20 15:57:35.551233 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:35.550773    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4"
Jul 20 15:57:35.551233 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:35.550798    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4" event={"ID":"fd3b5025-3574-410f-8634-e0858fc5e985","Type":"ContainerStarted","Data":"f9eae8a0dcff48a00c319abb0574ed7bf9422a41062b216b0b5637ee264544d6"}
Jul 20 15:57:35.588878 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:35.588822    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4" podStartSLOduration=1.58880361 podStartE2EDuration="1.58880361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:57:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:57:35.588038232 +0000 UTC m=+420.243301508" watchObservedRunningTime="2026-07-20 15:57:35.58880361 +0000 UTC m=+420.244066885"
Jul 20 15:57:35.594566 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:57:35.594530    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-thnxx\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:35.867717 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:57:35.867604    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-thnxx\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-thnxx"
Jul 20 15:57:35.869555 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:35.869523    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1af4d36-762f-4202-a3da-c30f67ea44a4" path="/var/lib/kubelet/pods/d1af4d36-762f-4202-a3da-c30f67ea44a4/volumes"
Jul 20 15:57:38.281887 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:38.281856    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:38.391708 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:38.391656    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:38.552113 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:38.552026    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:38.677780 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:38.677747    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:40.646008 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:40.645978    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:40.744421 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:40.744388    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:43.742744 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:43.742709    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:46.514009 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:46.513970    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:46.557972 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:46.557945    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jspg4"
Jul 20 15:57:46.583076 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:46.583046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:54.711882 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:54.711825    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:54.814091 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:54.814057    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:59.733234 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:59.733146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:59.828873 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:57:59.828831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:00.344668 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:00.344632    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:00.400727 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:00.400675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:04.125219 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:04.125180    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:04.223417 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:04.223385    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:14.568391 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:14.568348    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:15.235450 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:15.235420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:16.903242 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:16.903201    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:16.908596 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:16.908563    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"]
Jul 20 15:58:16.911857 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:16.911825    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:16.913852 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:16.913832    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 20 15:58:16.914093 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:16.914077    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vwqfz\""
Jul 20 15:58:16.920998 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:16.920963    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"]
Jul 20 15:58:16.998128 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:16.998097    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-config-file\") pod \"limitador-limitador-64c94fd64f-cf6z7\" (UID: \"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:16.998314 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:16.998136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56662\" (UniqueName: \"kubernetes.io/projected/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-kube-api-access-56662\") pod \"limitador-limitador-64c94fd64f-cf6z7\" (UID: \"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:17.016707 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.016654    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"]
Jul 20 15:58:17.029057 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.028971    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:17.090855 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.090821    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:17.098894 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.098861    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-config-file\") pod \"limitador-limitador-64c94fd64f-cf6z7\" (UID: \"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:17.099054 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.098903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56662\" (UniqueName: \"kubernetes.io/projected/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-kube-api-access-56662\") pod \"limitador-limitador-64c94fd64f-cf6z7\" (UID: \"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:17.099571 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.099552    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-config-file\") pod \"limitador-limitador-64c94fd64f-cf6z7\" (UID: \"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:17.113915 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.113886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56662\" (UniqueName: \"kubernetes.io/projected/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-kube-api-access-56662\") pod \"limitador-limitador-64c94fd64f-cf6z7\" (UID: \"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:17.222627 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.222590    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:17.357169 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.357145    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"]
Jul 20 15:58:17.359548 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:58:17.359520    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e7225b_f0d3_4a85_a4dc_72af6f3b81b0.slice/crio-d4f47935b145f465e395ec95880bf44bb2dc465e99fe9fb2596acfbf0fe839dd WatchSource:0}: Error finding container d4f47935b145f465e395ec95880bf44bb2dc465e99fe9fb2596acfbf0fe839dd: Status 404 returned error can't find the container with id d4f47935b145f465e395ec95880bf44bb2dc465e99fe9fb2596acfbf0fe839dd
Jul 20 15:58:17.723229 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.723139    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7" event={"ID":"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0","Type":"ContainerStarted","Data":"d4f47935b145f465e395ec95880bf44bb2dc465e99fe9fb2596acfbf0fe839dd"}
Jul 20 15:58:17.770847 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.770816    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:17.869609 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:17.869582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:19.815510 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:19.815466    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:20.738469 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:20.738433    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7" event={"ID":"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0","Type":"ContainerStarted","Data":"ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19"}
Jul 20 15:58:20.738672 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:20.738506    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:20.756283 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:20.756229    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7" podStartSLOduration=2.106236527 podStartE2EDuration="4.756209948s" totalImagesPullingTime="2.649973421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:58:20.755411261 +0000 UTC m=+465.410674537" watchObservedRunningTime="2026-07-20 15:58:20.756209948 +0000 UTC m=+465.411473225"
Jul 20 15:58:24.232273 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:24.232234    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:24.283126 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:24.283096    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:31.744529 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:31.744499    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:33.962491 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:33.962454    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"]
Jul 20 15:58:33.962969 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:33.962705    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7" podUID="31e7225b-f0d3-4a85-a4dc-72af6f3b81b0" containerName="limitador" containerID="cri-o://ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19" gracePeriod=30
Jul 20 15:58:34.513313 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.513285    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:34.635976 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.635879    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56662\" (UniqueName: \"kubernetes.io/projected/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-kube-api-access-56662\") pod \"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0\" (UID: \"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0\") "
Jul 20 15:58:34.635976 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.635972    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-config-file\") pod \"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0\" (UID: \"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0\") "
Jul 20 15:58:34.636384 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.636356    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-config-file" (OuterVolumeSpecName: "config-file") pod "31e7225b-f0d3-4a85-a4dc-72af6f3b81b0" (UID: "31e7225b-f0d3-4a85-a4dc-72af6f3b81b0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:58:34.638058 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.638034    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-kube-api-access-56662" (OuterVolumeSpecName: "kube-api-access-56662") pod "31e7225b-f0d3-4a85-a4dc-72af6f3b81b0" (UID: "31e7225b-f0d3-4a85-a4dc-72af6f3b81b0"). InnerVolumeSpecName "kube-api-access-56662". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:58:34.736915 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.736874    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56662\" (UniqueName: \"kubernetes.io/projected/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-kube-api-access-56662\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:58:34.736915 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.736908    2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0-config-file\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:58:34.794154 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.794121    2564 generic.go:356] "Generic (PLEG): container finished" podID="31e7225b-f0d3-4a85-a4dc-72af6f3b81b0" containerID="ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19" exitCode=0
Jul 20 15:58:34.794330 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.794167    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"
Jul 20 15:58:34.794330 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.794176    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7" event={"ID":"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0","Type":"ContainerDied","Data":"ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19"}
Jul 20 15:58:34.794330 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.794227    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cf6z7" event={"ID":"31e7225b-f0d3-4a85-a4dc-72af6f3b81b0","Type":"ContainerDied","Data":"d4f47935b145f465e395ec95880bf44bb2dc465e99fe9fb2596acfbf0fe839dd"}
Jul 20 15:58:34.794330 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.794253    2564 scope.go:117] "RemoveContainer" containerID="ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19"
Jul 20 15:58:34.803043 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.803026    2564 scope.go:117] "RemoveContainer" containerID="ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19"
Jul 20 15:58:34.803317 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:58:34.803295    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19\": container with ID starting with ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19 not found: ID does not exist" containerID="ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19"
Jul 20 15:58:34.803375 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.803326    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19"} err="failed to get container status \"ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19\": rpc error: code = NotFound desc = could not find container \"ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19\": container with ID starting with ba276dc131c69355c12235434db6845f24ea8bfd895a2a41d84e549a68881f19 not found: ID does not exist"
Jul 20 15:58:34.815734 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.815705    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"]
Jul 20 15:58:34.819639 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:34.819615    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cf6z7"]
Jul 20 15:58:35.868866 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:35.868837    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31e7225b-f0d3-4a85-a4dc-72af6f3b81b0" path="/var/lib/kubelet/pods/31e7225b-f0d3-4a85-a4dc-72af6f3b81b0/volumes"
Jul 20 15:58:38.343896 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.343863    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:38.432664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.432633    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:38.845544 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.845483    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"]
Jul 20 15:58:38.845782 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.845770    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31e7225b-f0d3-4a85-a4dc-72af6f3b81b0" containerName="limitador"
Jul 20 15:58:38.845834 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.845784    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="31e7225b-f0d3-4a85-a4dc-72af6f3b81b0" containerName="limitador"
Jul 20 15:58:38.845868 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.845839    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="31e7225b-f0d3-4a85-a4dc-72af6f3b81b0" containerName="limitador"
Jul 20 15:58:38.849980 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.849956    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"
Jul 20 15:58:38.851994 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.851969    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 15:58:38.852125 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.851969    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 20 15:58:38.852454 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.852436    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4cx6l\""
Jul 20 15:58:38.852545 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.852456    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 20 15:58:38.860959 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.860929    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"]
Jul 20 15:58:38.871602 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.871571    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:38.971647 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.971603    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/035ec57e-3ecf-4030-81d1-72d263f46989-data\") pod \"postgres-69fb4594cf-xh2wg\" (UID: \"035ec57e-3ecf-4030-81d1-72d263f46989\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"
Jul 20 15:58:38.971856 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:38.971830    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnfgm\" (UniqueName: \"kubernetes.io/projected/035ec57e-3ecf-4030-81d1-72d263f46989-kube-api-access-qnfgm\") pod \"postgres-69fb4594cf-xh2wg\" (UID: \"035ec57e-3ecf-4030-81d1-72d263f46989\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"
Jul 20 15:58:39.072971 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:39.072922    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/035ec57e-3ecf-4030-81d1-72d263f46989-data\") pod \"postgres-69fb4594cf-xh2wg\" (UID: \"035ec57e-3ecf-4030-81d1-72d263f46989\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"
Jul 20 15:58:39.073150 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:39.072992    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnfgm\" (UniqueName: \"kubernetes.io/projected/035ec57e-3ecf-4030-81d1-72d263f46989-kube-api-access-qnfgm\") pod \"postgres-69fb4594cf-xh2wg\" (UID: \"035ec57e-3ecf-4030-81d1-72d263f46989\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"
Jul 20 15:58:39.073316 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:39.073297    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/035ec57e-3ecf-4030-81d1-72d263f46989-data\") pod \"postgres-69fb4594cf-xh2wg\" (UID: \"035ec57e-3ecf-4030-81d1-72d263f46989\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"
Jul 20 15:58:39.081153 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:39.081117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnfgm\" (UniqueName: \"kubernetes.io/projected/035ec57e-3ecf-4030-81d1-72d263f46989-kube-api-access-qnfgm\") pod \"postgres-69fb4594cf-xh2wg\" (UID: \"035ec57e-3ecf-4030-81d1-72d263f46989\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"
Jul 20 15:58:39.161018 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:39.160924    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"
Jul 20 15:58:39.299327 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:39.299280    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"]
Jul 20 15:58:39.301472 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:58:39.301446    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035ec57e_3ecf_4030_81d1_72d263f46989.slice/crio-d10fad6a188a1c1a90afecb0ca1798d2c8456c31d5294fe9a7e303449730ec43 WatchSource:0}: Error finding container d10fad6a188a1c1a90afecb0ca1798d2c8456c31d5294fe9a7e303449730ec43: Status 404 returned error can't find the container with id d10fad6a188a1c1a90afecb0ca1798d2c8456c31d5294fe9a7e303449730ec43
Jul 20 15:58:39.813273 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:39.813238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg" event={"ID":"035ec57e-3ecf-4030-81d1-72d263f46989","Type":"ContainerStarted","Data":"d10fad6a188a1c1a90afecb0ca1798d2c8456c31d5294fe9a7e303449730ec43"}
Jul 20 15:58:44.837629 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:44.837585    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg" event={"ID":"035ec57e-3ecf-4030-81d1-72d263f46989","Type":"ContainerStarted","Data":"3afd2a327beb870404351b2f303a5d0829112797d3e0514cd2dcce146193d342"}
Jul 20 15:58:44.838177 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:44.837749    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"
Jul 20 15:58:44.852563 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:44.852507    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg" podStartSLOduration=2.10416753 podStartE2EDuration="6.852488788s" totalImagesPullingTime="4.748321258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:58:44.852334944 +0000 UTC m=+489.507598234" watchObservedRunningTime="2026-07-20 15:58:44.852488788 +0000 UTC m=+489.507752065"
Jul 20 15:58:50.871137 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:50.871108    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xh2wg"
Jul 20 15:58:51.193607 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:51.193513    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:51.590939 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:51.590905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:58.076867 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.076831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:58.122870 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.122831    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bb8c945f-vkj6j"]
Jul 20 15:58:58.145975 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.145939    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb8c945f-vkj6j"]
Jul 20 15:58:58.146128 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.146057    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:58:58.148948 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.148919    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 15:58:58.149101 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.148918    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 15:58:58.151350 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.151326    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-l4tv5\""
Jul 20 15:58:58.207386 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.207353    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:58.238593 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.238556    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-webhook-cert\") pod \"maas-controller-6bb8c945f-vkj6j\" (UID: \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\") " pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:58:58.238791 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.238601    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm2tz\" (UniqueName: \"kubernetes.io/projected/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-kube-api-access-tm2tz\") pod \"maas-controller-6bb8c945f-vkj6j\" (UID: \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\") " pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:58:58.258445 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.258412    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:58.320845 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.320805    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76d8696695-25rrq"]
Jul 20 15:58:58.323474 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.323441    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:58:58.335520 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.335493    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d8696695-25rrq"]
Jul 20 15:58:58.340082 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.340050    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-webhook-cert\") pod \"maas-controller-6bb8c945f-vkj6j\" (UID: \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\") " pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:58:58.340252 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.340097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm2tz\" (UniqueName: \"kubernetes.io/projected/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-kube-api-access-tm2tz\") pod \"maas-controller-6bb8c945f-vkj6j\" (UID: \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\") " pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:58:58.340252 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:58:58.340209    2564 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 20 15:58:58.340368 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:58:58.340282    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-webhook-cert podName:b9a451bc-fcb9-41be-99bc-bc3ab5ab9635 nodeName:}" failed. No retries permitted until 2026-07-20 15:58:58.840264962 +0000 UTC m=+503.495528232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-webhook-cert") pod "maas-controller-6bb8c945f-vkj6j" (UID: "b9a451bc-fcb9-41be-99bc-bc3ab5ab9635") : secret "maas-controller-webhook-cert" not found
Jul 20 15:58:58.352461 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.352426    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm2tz\" (UniqueName: \"kubernetes.io/projected/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-kube-api-access-tm2tz\") pod \"maas-controller-6bb8c945f-vkj6j\" (UID: \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\") " pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:58:58.440617 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.440581    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6861e5f-4bcb-463c-b213-66b1217b3a19-webhook-cert\") pod \"maas-controller-76d8696695-25rrq\" (UID: \"e6861e5f-4bcb-463c-b213-66b1217b3a19\") " pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:58:58.440820 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.440642    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfvm7\" (UniqueName: \"kubernetes.io/projected/e6861e5f-4bcb-463c-b213-66b1217b3a19-kube-api-access-cfvm7\") pod \"maas-controller-76d8696695-25rrq\" (UID: \"e6861e5f-4bcb-463c-b213-66b1217b3a19\") " pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:58:58.541335 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.541294    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6861e5f-4bcb-463c-b213-66b1217b3a19-webhook-cert\") pod \"maas-controller-76d8696695-25rrq\" (UID: \"e6861e5f-4bcb-463c-b213-66b1217b3a19\") " pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:58:58.541532 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.541356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvm7\" (UniqueName: \"kubernetes.io/projected/e6861e5f-4bcb-463c-b213-66b1217b3a19-kube-api-access-cfvm7\") pod \"maas-controller-76d8696695-25rrq\" (UID: \"e6861e5f-4bcb-463c-b213-66b1217b3a19\") " pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:58:58.541532 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:58:58.541476    2564 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 20 15:58:58.541628 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:58:58.541558    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6861e5f-4bcb-463c-b213-66b1217b3a19-webhook-cert podName:e6861e5f-4bcb-463c-b213-66b1217b3a19 nodeName:}" failed. No retries permitted until 2026-07-20 15:58:59.041537406 +0000 UTC m=+503.696800662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/e6861e5f-4bcb-463c-b213-66b1217b3a19-webhook-cert") pod "maas-controller-76d8696695-25rrq" (UID: "e6861e5f-4bcb-463c-b213-66b1217b3a19") : secret "maas-controller-webhook-cert" not found
Jul 20 15:58:58.549877 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.549844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfvm7\" (UniqueName: \"kubernetes.io/projected/e6861e5f-4bcb-463c-b213-66b1217b3a19-kube-api-access-cfvm7\") pod \"maas-controller-76d8696695-25rrq\" (UID: \"e6861e5f-4bcb-463c-b213-66b1217b3a19\") " pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:58:58.716876 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.716790    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:58.843878 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.843841    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-webhook-cert\") pod \"maas-controller-6bb8c945f-vkj6j\" (UID: \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\") " pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:58:58.846205 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:58.846181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-webhook-cert\") pod \"maas-controller-6bb8c945f-vkj6j\" (UID: \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\") " pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:58:59.044928 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:59.044889    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6861e5f-4bcb-463c-b213-66b1217b3a19-webhook-cert\") pod \"maas-controller-76d8696695-25rrq\" (UID: \"e6861e5f-4bcb-463c-b213-66b1217b3a19\") " pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:58:59.047315 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:59.047294    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6861e5f-4bcb-463c-b213-66b1217b3a19-webhook-cert\") pod \"maas-controller-76d8696695-25rrq\" (UID: \"e6861e5f-4bcb-463c-b213-66b1217b3a19\") " pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:58:59.058178 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:59.058145    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:58:59.198988 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:59.198954    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb8c945f-vkj6j"]
Jul 20 15:58:59.202279 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:58:59.202251    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a451bc_fcb9_41be_99bc_bc3ab5ab9635.slice/crio-453b072fbdce3e92f619cf0ceca8b8bdbf1dc87059852cc520a226632a40312d WatchSource:0}: Error finding container 453b072fbdce3e92f619cf0ceca8b8bdbf1dc87059852cc520a226632a40312d: Status 404 returned error can't find the container with id 453b072fbdce3e92f619cf0ceca8b8bdbf1dc87059852cc520a226632a40312d
Jul 20 15:58:59.233973 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:59.233938    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:58:59.383427 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:59.383392    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d8696695-25rrq"]
Jul 20 15:58:59.386278 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:58:59.386247    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6861e5f_4bcb_463c_b213_66b1217b3a19.slice/crio-ba9beb422e388da5e0fe6872e41296e636853e6d384c7d997e6f7ca31e2c5b24 WatchSource:0}: Error finding container ba9beb422e388da5e0fe6872e41296e636853e6d384c7d997e6f7ca31e2c5b24: Status 404 returned error can't find the container with id ba9beb422e388da5e0fe6872e41296e636853e6d384c7d997e6f7ca31e2c5b24
Jul 20 15:58:59.899151 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:59.899109    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb8c945f-vkj6j" event={"ID":"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635","Type":"ContainerStarted","Data":"453b072fbdce3e92f619cf0ceca8b8bdbf1dc87059852cc520a226632a40312d"}
Jul 20 15:58:59.900637 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:58:59.900608    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d8696695-25rrq" event={"ID":"e6861e5f-4bcb-463c-b213-66b1217b3a19","Type":"ContainerStarted","Data":"ba9beb422e388da5e0fe6872e41296e636853e6d384c7d997e6f7ca31e2c5b24"}
Jul 20 15:59:01.911562 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:01.911374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb8c945f-vkj6j" event={"ID":"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635","Type":"ContainerStarted","Data":"72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f"}
Jul 20 15:59:01.911562 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:01.911449    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:59:01.913607 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:01.913578    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d8696695-25rrq" event={"ID":"e6861e5f-4bcb-463c-b213-66b1217b3a19","Type":"ContainerStarted","Data":"685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac"}
Jul 20 15:59:01.913750 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:01.913644    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:59:01.931812 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:01.931761    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bb8c945f-vkj6j" podStartSLOduration=1.3449625219999999 podStartE2EDuration="3.931743818s" totalImagesPullingTime="2.586781296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:59:01.92945155 +0000 UTC m=+506.584714827" watchObservedRunningTime="2026-07-20 15:59:01.931743818 +0000 UTC m=+506.587007094"
Jul 20 15:59:01.948652 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:01.948611    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:01.960430 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:01.960382    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76d8696695-25rrq" podStartSLOduration=1.5673909670000001 podStartE2EDuration="3.960365568s" totalImagesPullingTime="2.392974601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:59:01.958972813 +0000 UTC m=+506.614236108" watchObservedRunningTime="2026-07-20 15:59:01.960365568 +0000 UTC m=+506.615628844"
Jul 20 15:59:02.013177 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:02.013142    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:12.924082 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:12.924049    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:59:12.924591 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:12.924208    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 15:59:12.994109 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:12.994072    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bb8c945f-vkj6j"]
Jul 20 15:59:12.994348 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:12.994298    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6bb8c945f-vkj6j" podUID="b9a451bc-fcb9-41be-99bc-bc3ab5ab9635" containerName="manager" containerID="cri-o://72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f" gracePeriod=30
Jul 20 15:59:13.244128 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.244102    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:59:13.277712 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.277666    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tm2tz\" (UniqueName: \"kubernetes.io/projected/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-kube-api-access-tm2tz\") pod \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\" (UID: \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\") "
Jul 20 15:59:13.277871 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.277761    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-webhook-cert\") pod \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\" (UID: \"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635\") "
Jul 20 15:59:13.279881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.279845    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-kube-api-access-tm2tz" (OuterVolumeSpecName: "kube-api-access-tm2tz") pod "b9a451bc-fcb9-41be-99bc-bc3ab5ab9635" (UID: "b9a451bc-fcb9-41be-99bc-bc3ab5ab9635"). InnerVolumeSpecName "kube-api-access-tm2tz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:59:13.279987 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.279857    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b9a451bc-fcb9-41be-99bc-bc3ab5ab9635" (UID: "b9a451bc-fcb9-41be-99bc-bc3ab5ab9635"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:59:13.378556 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.378518    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-webhook-cert\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:59:13.378556 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.378551    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tm2tz\" (UniqueName: \"kubernetes.io/projected/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635-kube-api-access-tm2tz\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 15:59:13.967709 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.967656    2564 generic.go:356] "Generic (PLEG): container finished" podID="b9a451bc-fcb9-41be-99bc-bc3ab5ab9635" containerID="72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f" exitCode=0
Jul 20 15:59:13.968107 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.967723    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb8c945f-vkj6j"
Jul 20 15:59:13.968107 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.967723    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb8c945f-vkj6j" event={"ID":"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635","Type":"ContainerDied","Data":"72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f"}
Jul 20 15:59:13.968107 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.967837    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb8c945f-vkj6j" event={"ID":"b9a451bc-fcb9-41be-99bc-bc3ab5ab9635","Type":"ContainerDied","Data":"453b072fbdce3e92f619cf0ceca8b8bdbf1dc87059852cc520a226632a40312d"}
Jul 20 15:59:13.968107 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.967858    2564 scope.go:117] "RemoveContainer" containerID="72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f"
Jul 20 15:59:13.975681 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.975662    2564 scope.go:117] "RemoveContainer" containerID="72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f"
Jul 20 15:59:13.975977 ip-10-0-134-220 kubenswrapper[2564]: E0720 15:59:13.975955    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f\": container with ID starting with 72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f not found: ID does not exist" containerID="72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f"
Jul 20 15:59:13.976029 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.975986    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f"} err="failed to get container status \"72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f\": rpc error: code = NotFound desc = could not find container \"72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f\": container with ID starting with 72ad4f97628560a17929bf77d382d0a5dd4efc89c431957473dbb41c432e821f not found: ID does not exist"
Jul 20 15:59:13.982881 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.982856    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bb8c945f-vkj6j"]
Jul 20 15:59:13.986962 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:13.986938    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bb8c945f-vkj6j"]
Jul 20 15:59:15.868574 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:15.868542    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9a451bc-fcb9-41be-99bc-bc3ab5ab9635" path="/var/lib/kubelet/pods/b9a451bc-fcb9-41be-99bc-bc3ab5ab9635/volumes"
Jul 20 15:59:33.629902 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:33.629798    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:33.726914 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:33.726880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:34.112951 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.112916    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:34.253142 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.253108    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"]
Jul 20 15:59:34.253405 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.253394    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9a451bc-fcb9-41be-99bc-bc3ab5ab9635" containerName="manager"
Jul 20 15:59:34.253467 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.253407    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9a451bc-fcb9-41be-99bc-bc3ab5ab9635" containerName="manager"
Jul 20 15:59:34.253520 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.253469    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9a451bc-fcb9-41be-99bc-bc3ab5ab9635" containerName="manager"
Jul 20 15:59:34.256345 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.256327    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.258664 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.258631    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 20 15:59:34.258889 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.258873    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7jrvn\""
Jul 20 15:59:34.268250 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.268221    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"]
Jul 20 15:59:34.333756 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.333722    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh8vx\" (UniqueName: \"kubernetes.io/projected/ce30f998-a0d9-431f-af09-2c5a7ac8b52d-kube-api-access-nh8vx\") pod \"maas-api-fbf94c8c9-wtqvz\" (UID: \"ce30f998-a0d9-431f-af09-2c5a7ac8b52d\") " pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.333939 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.333763    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce30f998-a0d9-431f-af09-2c5a7ac8b52d-openshift-service-ca\") pod \"maas-api-fbf94c8c9-wtqvz\" (UID: \"ce30f998-a0d9-431f-af09-2c5a7ac8b52d\") " pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.333939 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.333791    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce30f998-a0d9-431f-af09-2c5a7ac8b52d-maas-api-tls\") pod \"maas-api-fbf94c8c9-wtqvz\" (UID: \"ce30f998-a0d9-431f-af09-2c5a7ac8b52d\") " pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.412177 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.412061    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:34.434791 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.434756    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh8vx\" (UniqueName: \"kubernetes.io/projected/ce30f998-a0d9-431f-af09-2c5a7ac8b52d-kube-api-access-nh8vx\") pod \"maas-api-fbf94c8c9-wtqvz\" (UID: \"ce30f998-a0d9-431f-af09-2c5a7ac8b52d\") " pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.434965 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.434799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce30f998-a0d9-431f-af09-2c5a7ac8b52d-openshift-service-ca\") pod \"maas-api-fbf94c8c9-wtqvz\" (UID: \"ce30f998-a0d9-431f-af09-2c5a7ac8b52d\") " pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.434965 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.434830    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce30f998-a0d9-431f-af09-2c5a7ac8b52d-maas-api-tls\") pod \"maas-api-fbf94c8c9-wtqvz\" (UID: \"ce30f998-a0d9-431f-af09-2c5a7ac8b52d\") " pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.435432 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.435408    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce30f998-a0d9-431f-af09-2c5a7ac8b52d-openshift-service-ca\") pod \"maas-api-fbf94c8c9-wtqvz\" (UID: \"ce30f998-a0d9-431f-af09-2c5a7ac8b52d\") " pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.437211 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.437190    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce30f998-a0d9-431f-af09-2c5a7ac8b52d-maas-api-tls\") pod \"maas-api-fbf94c8c9-wtqvz\" (UID: \"ce30f998-a0d9-431f-af09-2c5a7ac8b52d\") " pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.446140 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.446105    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh8vx\" (UniqueName: \"kubernetes.io/projected/ce30f998-a0d9-431f-af09-2c5a7ac8b52d-kube-api-access-nh8vx\") pod \"maas-api-fbf94c8c9-wtqvz\" (UID: \"ce30f998-a0d9-431f-af09-2c5a7ac8b52d\") " pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.566645 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.566606    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:34.632865 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.632829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:34.714649 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.714455    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"]
Jul 20 15:59:34.717028 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.716996    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:34.717554 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:59:34.717527    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce30f998_a0d9_431f_af09_2c5a7ac8b52d.slice/crio-1e5786310ccffdf327ae496a38a8d65b4f6c787cc5ab4618f4ab09e250cddb5a WatchSource:0}: Error finding container 1e5786310ccffdf327ae496a38a8d65b4f6c787cc5ab4618f4ab09e250cddb5a: Status 404 returned error can't find the container with id 1e5786310ccffdf327ae496a38a8d65b4f6c787cc5ab4618f4ab09e250cddb5a
Jul 20 15:59:34.762401 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.762366    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-lfsj2"]
Jul 20 15:59:34.766788 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.766768    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2"
Jul 20 15:59:34.768898 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.768873    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 20 15:59:34.769012 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.768875    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-626wc\""
Jul 20 15:59:34.776262 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.776239    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-lfsj2"]
Jul 20 15:59:34.838319 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.838278    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9fvd\" (UniqueName: \"kubernetes.io/projected/805b55dd-0b56-4831-8165-29ef025d779a-kube-api-access-g9fvd\") pod \"payload-processing-5bf47cd988-lfsj2\" (UID: \"805b55dd-0b56-4831-8165-29ef025d779a\") " pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2"
Jul 20 15:59:34.838493 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.838337    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805b55dd-0b56-4831-8165-29ef025d779a-plugins-config-volume\") pod \"payload-processing-5bf47cd988-lfsj2\" (UID: \"805b55dd-0b56-4831-8165-29ef025d779a\") " pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2"
Jul 20 15:59:34.939586 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.939539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805b55dd-0b56-4831-8165-29ef025d779a-plugins-config-volume\") pod \"payload-processing-5bf47cd988-lfsj2\" (UID: \"805b55dd-0b56-4831-8165-29ef025d779a\") " pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2"
Jul 20 15:59:34.939777 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.939679    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9fvd\" (UniqueName: \"kubernetes.io/projected/805b55dd-0b56-4831-8165-29ef025d779a-kube-api-access-g9fvd\") pod \"payload-processing-5bf47cd988-lfsj2\" (UID: \"805b55dd-0b56-4831-8165-29ef025d779a\") " pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2"
Jul 20 15:59:34.940335 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.940305    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805b55dd-0b56-4831-8165-29ef025d779a-plugins-config-volume\") pod \"payload-processing-5bf47cd988-lfsj2\" (UID: \"805b55dd-0b56-4831-8165-29ef025d779a\") " pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2"
Jul 20 15:59:34.948409 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:34.948376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9fvd\" (UniqueName: \"kubernetes.io/projected/805b55dd-0b56-4831-8165-29ef025d779a-kube-api-access-g9fvd\") pod \"payload-processing-5bf47cd988-lfsj2\" (UID: \"805b55dd-0b56-4831-8165-29ef025d779a\") " pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2"
Jul 20 15:59:35.054191 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:35.054154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz" event={"ID":"ce30f998-a0d9-431f-af09-2c5a7ac8b52d","Type":"ContainerStarted","Data":"1e5786310ccffdf327ae496a38a8d65b4f6c787cc5ab4618f4ab09e250cddb5a"}
Jul 20 15:59:35.078036 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:35.077992    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2"
Jul 20 15:59:35.225151 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:35.225111    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-lfsj2"]
Jul 20 15:59:35.228912 ip-10-0-134-220 kubenswrapper[2564]: W0720 15:59:35.228879    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805b55dd_0b56_4831_8165_29ef025d779a.slice/crio-ce1897f4f3d7261958d9db5d4560639babd0f718213bf38fd45521e73f0f55c2 WatchSource:0}: Error finding container ce1897f4f3d7261958d9db5d4560639babd0f718213bf38fd45521e73f0f55c2: Status 404 returned error can't find the container with id ce1897f4f3d7261958d9db5d4560639babd0f718213bf38fd45521e73f0f55c2
Jul 20 15:59:36.063006 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:36.062903    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2" event={"ID":"805b55dd-0b56-4831-8165-29ef025d779a","Type":"ContainerStarted","Data":"ce1897f4f3d7261958d9db5d4560639babd0f718213bf38fd45521e73f0f55c2"}
Jul 20 15:59:37.069625 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:37.069585    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz" event={"ID":"ce30f998-a0d9-431f-af09-2c5a7ac8b52d","Type":"ContainerStarted","Data":"3433bb096f3989da83dd1bffed711e0b56f67d73d06c81d6a9e6a47fe95c6d03"}
Jul 20 15:59:37.070116 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:37.069816    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:37.088789 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:37.088727    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz" podStartSLOduration=1.4315628710000001 podStartE2EDuration="3.088679096s" totalImagesPullingTime="1.657116225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:59:37.086494511 +0000 UTC m=+541.741757798" watchObservedRunningTime="2026-07-20 15:59:37.088679096 +0000 UTC m=+541.743942372"
Jul 20 15:59:38.075605 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:38.075568    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2" event={"ID":"805b55dd-0b56-4831-8165-29ef025d779a","Type":"ContainerStarted","Data":"0795ad303df8eadf7c06bd4f0234c10b63b5122d5581e9f573689c30c2fd9aee"}
Jul 20 15:59:38.076062 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:38.075646    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2"
Jul 20 15:59:38.092332 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:38.092264    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2" podStartSLOduration=1.707326106 podStartE2EDuration="4.092244354s" totalImagesPullingTime="2.384918248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:59:38.091782987 +0000 UTC m=+542.747046263" watchObservedRunningTime="2026-07-20 15:59:38.092244354 +0000 UTC m=+542.747507630"
Jul 20 15:59:43.081262 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:43.081231    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-fbf94c8c9-wtqvz"
Jul 20 15:59:44.780701 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:44.780658    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:44.875575 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:44.875542    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:49.082558 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:49.082517    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-lfsj2"
Jul 20 15:59:52.827513 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:52.827477    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:59.569221 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.569185    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"]
Jul 20 15:59:59.573095 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.573070    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.575856 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.575828    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 20 15:59:59.576014 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.575854    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 20 15:59:59.576240 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.576225    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 20 15:59:59.576385 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.576261    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lpsx4\""
Jul 20 15:59:59.584883 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.584861    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"]
Jul 20 15:59:59.593568 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.593539    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:59.654669 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.654633    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.654872 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.654759    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.654872 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.654782    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.654872 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.654809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0280a46a-5bea-4910-a0e0-bcc00b2380b4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.654872 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.654829    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.654872 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.654846    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv4lm\" (UniqueName: \"kubernetes.io/projected/0280a46a-5bea-4910-a0e0-bcc00b2380b4-kube-api-access-dv4lm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.755394 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.755352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0280a46a-5bea-4910-a0e0-bcc00b2380b4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.755394 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.755403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.755660 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.755420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv4lm\" (UniqueName: \"kubernetes.io/projected/0280a46a-5bea-4910-a0e0-bcc00b2380b4-kube-api-access-dv4lm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.755660 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.755444    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.755660 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.755515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.755660 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.755541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.755883 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.755845    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.755919 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.755900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.756023 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.755992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.757941 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.757919    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0280a46a-5bea-4910-a0e0-bcc00b2380b4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.758162 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.758142    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0280a46a-5bea-4910-a0e0-bcc00b2380b4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.763389 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.763364    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv4lm\" (UniqueName: \"kubernetes.io/projected/0280a46a-5bea-4910-a0e0-bcc00b2380b4-kube-api-access-dv4lm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b8ggb\" (UID: \"0280a46a-5bea-4910-a0e0-bcc00b2380b4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 15:59:59.884652 ip-10-0-134-220 kubenswrapper[2564]: I0720 15:59:59.884564    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 16:00:00.020733 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:00.020670    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"]
Jul 20 16:00:00.024725 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:00:00.024669    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0280a46a_5bea_4910_a0e0_bcc00b2380b4.slice/crio-7169905e3f620ff89f69acce74e9ca7f576c24e3cd278bb441e49beb0f776802 WatchSource:0}: Error finding container 7169905e3f620ff89f69acce74e9ca7f576c24e3cd278bb441e49beb0f776802: Status 404 returned error can't find the container with id 7169905e3f620ff89f69acce74e9ca7f576c24e3cd278bb441e49beb0f776802
Jul 20 16:00:00.166144 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:00.166055    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb" event={"ID":"0280a46a-5bea-4910-a0e0-bcc00b2380b4","Type":"ContainerStarted","Data":"7169905e3f620ff89f69acce74e9ca7f576c24e3cd278bb441e49beb0f776802"}
Jul 20 16:00:02.176913 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:02.176872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb" event={"ID":"0280a46a-5bea-4910-a0e0-bcc00b2380b4","Type":"ContainerStarted","Data":"bfd81e8602c0e581d346354b4089855057d709351a6efd4d34efe623b022b078"}
Jul 20 16:00:02.195844 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:02.195788    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb" podStartSLOduration=2.050558362 podStartE2EDuration="3.195770911s" totalImagesPullingTime="1.145212549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:00:02.194061467 +0000 UTC m=+566.849324745" watchObservedRunningTime="2026-07-20 16:00:02.195770911 +0000 UTC m=+566.851034186"
Jul 20 16:00:08.991857 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:08.991819    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:09.885429 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:09.885390    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 16:00:09.885609 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:09.885461    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 16:00:09.898036 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:09.898007    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 16:00:10.220351 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:10.220266    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b8ggb"
Jul 20 16:00:14.969251 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:14.969211    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"]
Jul 20 16:00:14.975918 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:14.975894    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:14.978714 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:14.978671    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 20 16:00:14.982641 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:14.982604    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptmtc\" (UniqueName: \"kubernetes.io/projected/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-kube-api-access-ptmtc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:14.982790 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:14.982665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:14.982790 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:14.982749    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:14.982790 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:14.982781    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:14.983294 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:14.982821    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:14.983294 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:14.982844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:14.988711 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:14.988658    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"]
Jul 20 16:00:15.015179 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.015144    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:15.083862 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.083820    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptmtc\" (UniqueName: \"kubernetes.io/projected/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-kube-api-access-ptmtc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.084060 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.083882    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.084060 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.083962    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.084060 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.083990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.084060 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.084029    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.084247 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.084080    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.084486 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.084457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.084627 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.084528    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.084627 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.084558    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.086232 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.086212    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.086657 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.086639    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.093107 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.093081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptmtc\" (UniqueName: \"kubernetes.io/projected/ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10-kube-api-access-ptmtc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm\" (UID: \"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.286481 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.286436    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:15.423888 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:15.423709    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"]
Jul 20 16:00:15.426460 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:00:15.426434    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0f2ac0_2343_49ff_9d28_bcd8f0e38a10.slice/crio-ce2f371fa353b4b13085d00a2021e197b9f0e22934c6e09af3d41a240c7c4f30 WatchSource:0}: Error finding container ce2f371fa353b4b13085d00a2021e197b9f0e22934c6e09af3d41a240c7c4f30: Status 404 returned error can't find the container with id ce2f371fa353b4b13085d00a2021e197b9f0e22934c6e09af3d41a240c7c4f30
Jul 20 16:00:16.236231 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:16.236189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm" event={"ID":"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10","Type":"ContainerStarted","Data":"ce2f371fa353b4b13085d00a2021e197b9f0e22934c6e09af3d41a240c7c4f30"}
Jul 20 16:00:20.255770 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:20.255733    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm" event={"ID":"ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10","Type":"ContainerStarted","Data":"483a3251cdc7b47ae963d91564e720e7ec9dd520f1e096dcac6c13e64e95c324"}
Jul 20 16:00:20.282499 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:20.282441    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm" podStartSLOduration=2.123265738 podStartE2EDuration="6.282425734s" totalImagesPullingTime="4.159159996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:00:20.282173372 +0000 UTC m=+584.937436649" watchObservedRunningTime="2026-07-20 16:00:20.282425734 +0000 UTC m=+584.937689009"
Jul 20 16:00:20.465454 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:20.465423    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:20.867073 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:20.867000    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:25.287428 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:25.287391    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:25.287878 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:25.287448    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:25.300001 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:25.299971    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:26.291265 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:26.291232    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm"
Jul 20 16:00:35.790095 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:35.790063    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:00:35.792281 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:35.792262    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:00:45.299734 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:45.299658    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:58.056961 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:00:58.056870    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:58.345292 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:01:58.345254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:05.559958 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:02:05.559920    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:05.665370 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:02:05.665334    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:13.840887 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:02:13.840848    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:13.894725 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:02:13.894680    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:14.003928 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:02:14.003893    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:15.033048 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:02:15.033014    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:15.138596 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:02:15.138564    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:53.094226 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:02:53.094185    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:04.347577 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.347539    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76d8696695-25rrq"]
Jul 20 16:03:04.348051 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.347741    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-76d8696695-25rrq" podUID="e6861e5f-4bcb-463c-b213-66b1217b3a19" containerName="manager" containerID="cri-o://685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac" gracePeriod=30
Jul 20 16:03:04.586097 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.586065    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 16:03:04.625812 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.625718    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfvm7\" (UniqueName: \"kubernetes.io/projected/e6861e5f-4bcb-463c-b213-66b1217b3a19-kube-api-access-cfvm7\") pod \"e6861e5f-4bcb-463c-b213-66b1217b3a19\" (UID: \"e6861e5f-4bcb-463c-b213-66b1217b3a19\") "
Jul 20 16:03:04.625812 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.625805    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6861e5f-4bcb-463c-b213-66b1217b3a19-webhook-cert\") pod \"e6861e5f-4bcb-463c-b213-66b1217b3a19\" (UID: \"e6861e5f-4bcb-463c-b213-66b1217b3a19\") "
Jul 20 16:03:04.627883 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.627849    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6861e5f-4bcb-463c-b213-66b1217b3a19-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e6861e5f-4bcb-463c-b213-66b1217b3a19" (UID: "e6861e5f-4bcb-463c-b213-66b1217b3a19"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:03:04.628006 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.627920    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6861e5f-4bcb-463c-b213-66b1217b3a19-kube-api-access-cfvm7" (OuterVolumeSpecName: "kube-api-access-cfvm7") pod "e6861e5f-4bcb-463c-b213-66b1217b3a19" (UID: "e6861e5f-4bcb-463c-b213-66b1217b3a19"). InnerVolumeSpecName "kube-api-access-cfvm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:03:04.726533 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.726492    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfvm7\" (UniqueName: \"kubernetes.io/projected/e6861e5f-4bcb-463c-b213-66b1217b3a19-kube-api-access-cfvm7\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:03:04.726533 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.726525    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6861e5f-4bcb-463c-b213-66b1217b3a19-webhook-cert\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:03:04.957156 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.957062    2564 generic.go:356] "Generic (PLEG): container finished" podID="e6861e5f-4bcb-463c-b213-66b1217b3a19" containerID="685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac" exitCode=0
Jul 20 16:03:04.957156 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.957105    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d8696695-25rrq"
Jul 20 16:03:04.957156 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.957143    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d8696695-25rrq" event={"ID":"e6861e5f-4bcb-463c-b213-66b1217b3a19","Type":"ContainerDied","Data":"685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac"}
Jul 20 16:03:04.957372 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.957194    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d8696695-25rrq" event={"ID":"e6861e5f-4bcb-463c-b213-66b1217b3a19","Type":"ContainerDied","Data":"ba9beb422e388da5e0fe6872e41296e636853e6d384c7d997e6f7ca31e2c5b24"}
Jul 20 16:03:04.957372 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.957220    2564 scope.go:117] "RemoveContainer" containerID="685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac"
Jul 20 16:03:04.966003 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.965981    2564 scope.go:117] "RemoveContainer" containerID="685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac"
Jul 20 16:03:04.966313 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:03:04.966294    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac\": container with ID starting with 685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac not found: ID does not exist" containerID="685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac"
Jul 20 16:03:04.966375 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.966323    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac"} err="failed to get container status \"685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac\": rpc error: code = NotFound desc = could not find container \"685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac\": container with ID starting with 685e036a16d8f4a71670092fe5dd6f1ca4a904321591a1f19a5259fa6877f1ac not found: ID does not exist"
Jul 20 16:03:04.977815 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.977778    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76d8696695-25rrq"]
Jul 20 16:03:04.983893 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:04.983864    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76d8696695-25rrq"]
Jul 20 16:03:05.874198 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:03:05.874167    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6861e5f-4bcb-463c-b213-66b1217b3a19" path="/var/lib/kubelet/pods/e6861e5f-4bcb-463c-b213-66b1217b3a19/volumes"
Jul 20 16:04:05.808106 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.808023    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76d8696695-plcrz"]
Jul 20 16:04:05.808550 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.808344    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6861e5f-4bcb-463c-b213-66b1217b3a19" containerName="manager"
Jul 20 16:04:05.808550 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.808356    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6861e5f-4bcb-463c-b213-66b1217b3a19" containerName="manager"
Jul 20 16:04:05.808550 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.808404    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6861e5f-4bcb-463c-b213-66b1217b3a19" containerName="manager"
Jul 20 16:04:05.810306 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.810290    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:05.812772 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.812750    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-l4tv5\""
Jul 20 16:04:05.812772 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.812770    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 16:04:05.812920 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.812771    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 16:04:05.819966 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.819927    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d8696695-plcrz"]
Jul 20 16:04:05.845197 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.845169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/21530517-fe25-4357-b1cd-f07ff47dcbb6-webhook-cert\") pod \"maas-controller-76d8696695-plcrz\" (UID: \"21530517-fe25-4357-b1cd-f07ff47dcbb6\") " pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:05.845383 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.845208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2d28\" (UniqueName: \"kubernetes.io/projected/21530517-fe25-4357-b1cd-f07ff47dcbb6-kube-api-access-h2d28\") pod \"maas-controller-76d8696695-plcrz\" (UID: \"21530517-fe25-4357-b1cd-f07ff47dcbb6\") " pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:05.945851 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.945806    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/21530517-fe25-4357-b1cd-f07ff47dcbb6-webhook-cert\") pod \"maas-controller-76d8696695-plcrz\" (UID: \"21530517-fe25-4357-b1cd-f07ff47dcbb6\") " pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:05.945851 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.945855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2d28\" (UniqueName: \"kubernetes.io/projected/21530517-fe25-4357-b1cd-f07ff47dcbb6-kube-api-access-h2d28\") pod \"maas-controller-76d8696695-plcrz\" (UID: \"21530517-fe25-4357-b1cd-f07ff47dcbb6\") " pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:05.948234 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.948207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/21530517-fe25-4357-b1cd-f07ff47dcbb6-webhook-cert\") pod \"maas-controller-76d8696695-plcrz\" (UID: \"21530517-fe25-4357-b1cd-f07ff47dcbb6\") " pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:05.954641 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:05.954613    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2d28\" (UniqueName: \"kubernetes.io/projected/21530517-fe25-4357-b1cd-f07ff47dcbb6-kube-api-access-h2d28\") pod \"maas-controller-76d8696695-plcrz\" (UID: \"21530517-fe25-4357-b1cd-f07ff47dcbb6\") " pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:06.121037 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:06.120951    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:06.256838 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:06.256803    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d8696695-plcrz"]
Jul 20 16:04:06.259165 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:04:06.259129    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21530517_fe25_4357_b1cd_f07ff47dcbb6.slice/crio-38404375d5bbbc3246211e82fa95f80d93d5cec060646ec9e471f95d6cb2d4f4 WatchSource:0}: Error finding container 38404375d5bbbc3246211e82fa95f80d93d5cec060646ec9e471f95d6cb2d4f4: Status 404 returned error can't find the container with id 38404375d5bbbc3246211e82fa95f80d93d5cec060646ec9e471f95d6cb2d4f4
Jul 20 16:04:06.260428 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:06.260406    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:04:07.225937 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:07.225896    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d8696695-plcrz" event={"ID":"21530517-fe25-4357-b1cd-f07ff47dcbb6","Type":"ContainerStarted","Data":"d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb"}
Jul 20 16:04:07.226345 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:07.225964    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d8696695-plcrz" event={"ID":"21530517-fe25-4357-b1cd-f07ff47dcbb6","Type":"ContainerStarted","Data":"38404375d5bbbc3246211e82fa95f80d93d5cec060646ec9e471f95d6cb2d4f4"}
Jul 20 16:04:07.226345 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:07.225993    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:07.246175 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:07.246100    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76d8696695-plcrz" podStartSLOduration=1.769844891 podStartE2EDuration="2.246082135s" totalImagesPullingTime="476.237244ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:04:07.245066963 +0000 UTC m=+811.900330239" watchObservedRunningTime="2026-07-20 16:04:07.246082135 +0000 UTC m=+811.901345442"
Jul 20 16:04:18.232820 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:18.232791    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:22.804996 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:22.804948    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:22.915723 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:22.915667    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76d8696695-plcrz"]
Jul 20 16:04:22.915908 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:22.915880    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-76d8696695-plcrz" podUID="21530517-fe25-4357-b1cd-f07ff47dcbb6" containerName="manager" containerID="cri-o://d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb" gracePeriod=30
Jul 20 16:04:23.162219 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.162196    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:23.294275 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.294229    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2d28\" (UniqueName: \"kubernetes.io/projected/21530517-fe25-4357-b1cd-f07ff47dcbb6-kube-api-access-h2d28\") pod \"21530517-fe25-4357-b1cd-f07ff47dcbb6\" (UID: \"21530517-fe25-4357-b1cd-f07ff47dcbb6\") "
Jul 20 16:04:23.294472 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.294301    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/21530517-fe25-4357-b1cd-f07ff47dcbb6-webhook-cert\") pod \"21530517-fe25-4357-b1cd-f07ff47dcbb6\" (UID: \"21530517-fe25-4357-b1cd-f07ff47dcbb6\") "
Jul 20 16:04:23.295879 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.295847    2564 generic.go:356] "Generic (PLEG): container finished" podID="21530517-fe25-4357-b1cd-f07ff47dcbb6" containerID="d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb" exitCode=0
Jul 20 16:04:23.295993 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.295909    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d8696695-plcrz"
Jul 20 16:04:23.295993 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.295928    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d8696695-plcrz" event={"ID":"21530517-fe25-4357-b1cd-f07ff47dcbb6","Type":"ContainerDied","Data":"d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb"}
Jul 20 16:04:23.295993 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.295982    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d8696695-plcrz" event={"ID":"21530517-fe25-4357-b1cd-f07ff47dcbb6","Type":"ContainerDied","Data":"38404375d5bbbc3246211e82fa95f80d93d5cec060646ec9e471f95d6cb2d4f4"}
Jul 20 16:04:23.296153 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.296004    2564 scope.go:117] "RemoveContainer" containerID="d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb"
Jul 20 16:04:23.296580 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.296552    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21530517-fe25-4357-b1cd-f07ff47dcbb6-kube-api-access-h2d28" (OuterVolumeSpecName: "kube-api-access-h2d28") pod "21530517-fe25-4357-b1cd-f07ff47dcbb6" (UID: "21530517-fe25-4357-b1cd-f07ff47dcbb6"). InnerVolumeSpecName "kube-api-access-h2d28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:04:23.296670 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.296605    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21530517-fe25-4357-b1cd-f07ff47dcbb6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "21530517-fe25-4357-b1cd-f07ff47dcbb6" (UID: "21530517-fe25-4357-b1cd-f07ff47dcbb6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:04:23.311420 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.311394    2564 scope.go:117] "RemoveContainer" containerID="d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb"
Jul 20 16:04:23.311758 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:04:23.311730    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb\": container with ID starting with d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb not found: ID does not exist" containerID="d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb"
Jul 20 16:04:23.311863 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.311771    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb"} err="failed to get container status \"d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb\": rpc error: code = NotFound desc = could not find container \"d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb\": container with ID starting with d1f7d47a22e5e6544ac1ace42a5eca60f15ba888d4130194724f15e350cf8ecb not found: ID does not exist"
Jul 20 16:04:23.395411 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.395319    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/21530517-fe25-4357-b1cd-f07ff47dcbb6-webhook-cert\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:04:23.395411 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.395354    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2d28\" (UniqueName: \"kubernetes.io/projected/21530517-fe25-4357-b1cd-f07ff47dcbb6-kube-api-access-h2d28\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:04:23.623504 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.623471    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76d8696695-plcrz"]
Jul 20 16:04:23.624925 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.624903    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76d8696695-plcrz"]
Jul 20 16:04:23.870361 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:23.870330    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21530517-fe25-4357-b1cd-f07ff47dcbb6" path="/var/lib/kubelet/pods/21530517-fe25-4357-b1cd-f07ff47dcbb6/volumes"
Jul 20 16:04:24.332356 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.332322    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:24.466548 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.466514    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76d8696695-zq55p"]
Jul 20 16:04:24.466843 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.466832    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21530517-fe25-4357-b1cd-f07ff47dcbb6" containerName="manager"
Jul 20 16:04:24.466907 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.466845    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="21530517-fe25-4357-b1cd-f07ff47dcbb6" containerName="manager"
Jul 20 16:04:24.466956 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.466907    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="21530517-fe25-4357-b1cd-f07ff47dcbb6" containerName="manager"
Jul 20 16:04:24.475467 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.475434    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d8696695-zq55p"
Jul 20 16:04:24.480357 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.480331    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-l4tv5\""
Jul 20 16:04:24.480952 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.480922    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 16:04:24.481082 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.480958    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 16:04:24.491807 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.491772    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d8696695-zq55p"]
Jul 20 16:04:24.606312 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.606213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba52a7b9-3b00-4304-9891-a43d97f47cf4-webhook-cert\") pod \"maas-controller-76d8696695-zq55p\" (UID: \"ba52a7b9-3b00-4304-9891-a43d97f47cf4\") " pod="opendatahub/maas-controller-76d8696695-zq55p"
Jul 20 16:04:24.606312 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.606262    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgg4d\" (UniqueName: \"kubernetes.io/projected/ba52a7b9-3b00-4304-9891-a43d97f47cf4-kube-api-access-cgg4d\") pod \"maas-controller-76d8696695-zq55p\" (UID: \"ba52a7b9-3b00-4304-9891-a43d97f47cf4\") " pod="opendatahub/maas-controller-76d8696695-zq55p"
Jul 20 16:04:24.706772 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.706734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba52a7b9-3b00-4304-9891-a43d97f47cf4-webhook-cert\") pod \"maas-controller-76d8696695-zq55p\" (UID: \"ba52a7b9-3b00-4304-9891-a43d97f47cf4\") " pod="opendatahub/maas-controller-76d8696695-zq55p"
Jul 20 16:04:24.706945 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.706780    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgg4d\" (UniqueName: \"kubernetes.io/projected/ba52a7b9-3b00-4304-9891-a43d97f47cf4-kube-api-access-cgg4d\") pod \"maas-controller-76d8696695-zq55p\" (UID: \"ba52a7b9-3b00-4304-9891-a43d97f47cf4\") " pod="opendatahub/maas-controller-76d8696695-zq55p"
Jul 20 16:04:24.709172 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.709153    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba52a7b9-3b00-4304-9891-a43d97f47cf4-webhook-cert\") pod \"maas-controller-76d8696695-zq55p\" (UID: \"ba52a7b9-3b00-4304-9891-a43d97f47cf4\") " pod="opendatahub/maas-controller-76d8696695-zq55p"
Jul 20 16:04:24.720054 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.720024    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgg4d\" (UniqueName: \"kubernetes.io/projected/ba52a7b9-3b00-4304-9891-a43d97f47cf4-kube-api-access-cgg4d\") pod \"maas-controller-76d8696695-zq55p\" (UID: \"ba52a7b9-3b00-4304-9891-a43d97f47cf4\") " pod="opendatahub/maas-controller-76d8696695-zq55p"
Jul 20 16:04:24.786186 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.786140    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d8696695-zq55p"
Jul 20 16:04:24.927663 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:24.927636    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d8696695-zq55p"]
Jul 20 16:04:24.930433 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:04:24.930391    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba52a7b9_3b00_4304_9891_a43d97f47cf4.slice/crio-733adc9827a07988eb6926fdd30d106385bbbc1155200413530d9cfaa7145c70 WatchSource:0}: Error finding container 733adc9827a07988eb6926fdd30d106385bbbc1155200413530d9cfaa7145c70: Status 404 returned error can't find the container with id 733adc9827a07988eb6926fdd30d106385bbbc1155200413530d9cfaa7145c70
Jul 20 16:04:25.306971 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:25.306935    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d8696695-zq55p" event={"ID":"ba52a7b9-3b00-4304-9891-a43d97f47cf4","Type":"ContainerStarted","Data":"733adc9827a07988eb6926fdd30d106385bbbc1155200413530d9cfaa7145c70"}
Jul 20 16:04:26.312336 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:26.312299    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d8696695-zq55p" event={"ID":"ba52a7b9-3b00-4304-9891-a43d97f47cf4","Type":"ContainerStarted","Data":"8ebbcc2a5d2cd5f7ac5883fa697bd6acbee3f81cd913854d48a4147077a70af4"}
Jul 20 16:04:26.312820 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:26.312418    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76d8696695-zq55p"
Jul 20 16:04:26.339032 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:26.338969    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76d8696695-zq55p" podStartSLOduration=1.77404881 podStartE2EDuration="2.338950745s" totalImagesPullingTime="564.901935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:04:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:04:26.337097809 +0000 UTC m=+830.992361085" watchObservedRunningTime="2026-07-20 16:04:26.338950745 +0000 UTC m=+830.994214021"
Jul 20 16:04:37.319582 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:04:37.319545    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76d8696695-zq55p"
Jul 20 16:05:03.356406 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:03.356367    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:03.479987 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:03.479952    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:03.764609 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:03.764529    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:10.206790 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:10.206747    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:10.318194 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:10.318160    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:10.516983 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:10.516896    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:10.684660 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:10.684624    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:17.571326 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:17.571289    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:17.775045 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:17.775012    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:35.814573 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:35.814500    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:05:35.815040 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:05:35.814906    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:06:34.637154 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:06:34.637118    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:08:24.140072 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:08:24.140036    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:09:12.224858 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:09:12.224823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:09:12.281594 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:09:12.281557    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:35.837100 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:10:35.837071    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:10:35.838063 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:10:35.838042    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:13:14.640604 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:14.640526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:13:28.381113 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.381065    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"]
Jul 20 16:13:28.384500 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.384479    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.386826 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.386803    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-5d2e68d4-kserve-self-signed-certs\""
Jul 20 16:13:28.400971 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.400943    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"]
Jul 20 16:13:28.437724 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.437674    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:13:28.473884 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.473850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-tmp-dir\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.474049 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.473914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-model-cache\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.474049 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.473998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxlsr\" (UniqueName: \"kubernetes.io/projected/acb8e21f-a098-4e39-9926-e5544d26f373-kube-api-access-wxlsr\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.474049 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.474039    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-dshm\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.474204 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.474057    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-home\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.474204 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.474151    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acb8e21f-a098-4e39-9926-e5544d26f373-tls-certs\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.574582 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.574541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-model-cache\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.574778 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.574605    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxlsr\" (UniqueName: \"kubernetes.io/projected/acb8e21f-a098-4e39-9926-e5544d26f373-kube-api-access-wxlsr\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.574778 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.574643    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-dshm\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.574778 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.574666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-home\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.574778 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.574757    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acb8e21f-a098-4e39-9926-e5544d26f373-tls-certs\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.575070 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.574792    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-tmp-dir\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.575070 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.575056    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-model-cache\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.575174 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.575124    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-home\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.575174 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.575148    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-tmp-dir\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.577249 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.577218    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-dshm\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.577401 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.577379    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acb8e21f-a098-4e39-9926-e5544d26f373-tls-certs\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.583640 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.583615    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxlsr\" (UniqueName: \"kubernetes.io/projected/acb8e21f-a098-4e39-9926-e5544d26f373-kube-api-access-wxlsr\") pod \"e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") " pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.694544 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.694452    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:28.835626 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:28.835589    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"]
Jul 20 16:13:28.838997 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:13:28.838965    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacb8e21f_a098_4e39_9926_e5544d26f373.slice/crio-071ee793366b4e6e2e4ba25eb1359547c9e392a66df4a5ad90206b6896d24424 WatchSource:0}: Error finding container 071ee793366b4e6e2e4ba25eb1359547c9e392a66df4a5ad90206b6896d24424: Status 404 returned error can't find the container with id 071ee793366b4e6e2e4ba25eb1359547c9e392a66df4a5ad90206b6896d24424
Jul 20 16:13:29.588933 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:29.588897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl" event={"ID":"acb8e21f-a098-4e39-9926-e5544d26f373","Type":"ContainerStarted","Data":"284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b"}
Jul 20 16:13:29.589320 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:29.588940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl" event={"ID":"acb8e21f-a098-4e39-9926-e5544d26f373","Type":"ContainerStarted","Data":"071ee793366b4e6e2e4ba25eb1359547c9e392a66df4a5ad90206b6896d24424"}
Jul 20 16:13:29.604584 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:29.604515    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl" podStartSLOduration=1.604469617 podStartE2EDuration="1.604469617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:13:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:13:29.604035513 +0000 UTC m=+1374.259298789" watchObservedRunningTime="2026-07-20 16:13:29.604469617 +0000 UTC m=+1374.259732894"
Jul 20 16:13:38.694981 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:38.694933    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:38.695392 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:38.695003    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:38.707773 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:38.707746    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:39.643086 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:39.643052    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:13:51.681117 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:51.681064    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:13:52.688761 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:52.688726    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"]
Jul 20 16:13:52.700007 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:13:52.699977    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:14:07.710895 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:07.710822    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl" podUID="acb8e21f-a098-4e39-9926-e5544d26f373" containerName="main" containerID="cri-o://284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b" gracePeriod=45
Jul 20 16:14:07.955436 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:07.955411    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:14:08.032243 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.032204    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-dshm\") pod \"acb8e21f-a098-4e39-9926-e5544d26f373\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") "
Jul 20 16:14:08.032243 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.032245    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-home\") pod \"acb8e21f-a098-4e39-9926-e5544d26f373\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") "
Jul 20 16:14:08.032514 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.032299    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acb8e21f-a098-4e39-9926-e5544d26f373-tls-certs\") pod \"acb8e21f-a098-4e39-9926-e5544d26f373\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") "
Jul 20 16:14:08.032514 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.032331    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-tmp-dir\") pod \"acb8e21f-a098-4e39-9926-e5544d26f373\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") "
Jul 20 16:14:08.032514 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.032370    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-model-cache\") pod \"acb8e21f-a098-4e39-9926-e5544d26f373\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") "
Jul 20 16:14:08.032514 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.032417    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxlsr\" (UniqueName: \"kubernetes.io/projected/acb8e21f-a098-4e39-9926-e5544d26f373-kube-api-access-wxlsr\") pod \"acb8e21f-a098-4e39-9926-e5544d26f373\" (UID: \"acb8e21f-a098-4e39-9926-e5544d26f373\") "
Jul 20 16:14:08.032746 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.032613    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-home" (OuterVolumeSpecName: "home") pod "acb8e21f-a098-4e39-9926-e5544d26f373" (UID: "acb8e21f-a098-4e39-9926-e5544d26f373"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:14:08.032746 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.032642    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "acb8e21f-a098-4e39-9926-e5544d26f373" (UID: "acb8e21f-a098-4e39-9926-e5544d26f373"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:14:08.032746 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.032721    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-model-cache" (OuterVolumeSpecName: "model-cache") pod "acb8e21f-a098-4e39-9926-e5544d26f373" (UID: "acb8e21f-a098-4e39-9926-e5544d26f373"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:14:08.034827 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.034799    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acb8e21f-a098-4e39-9926-e5544d26f373-kube-api-access-wxlsr" (OuterVolumeSpecName: "kube-api-access-wxlsr") pod "acb8e21f-a098-4e39-9926-e5544d26f373" (UID: "acb8e21f-a098-4e39-9926-e5544d26f373"). InnerVolumeSpecName "kube-api-access-wxlsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:14:08.034946 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.034825    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acb8e21f-a098-4e39-9926-e5544d26f373-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "acb8e21f-a098-4e39-9926-e5544d26f373" (UID: "acb8e21f-a098-4e39-9926-e5544d26f373"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:14:08.034946 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.034906    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-dshm" (OuterVolumeSpecName: "dshm") pod "acb8e21f-a098-4e39-9926-e5544d26f373" (UID: "acb8e21f-a098-4e39-9926-e5544d26f373"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:14:08.133429 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.133396    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-tmp-dir\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:14:08.133429 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.133425    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-model-cache\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:14:08.133429 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.133434    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxlsr\" (UniqueName: \"kubernetes.io/projected/acb8e21f-a098-4e39-9926-e5544d26f373-kube-api-access-wxlsr\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:14:08.133680 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.133442    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-dshm\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:14:08.133680 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.133452    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acb8e21f-a098-4e39-9926-e5544d26f373-home\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:14:08.133680 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.133459    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acb8e21f-a098-4e39-9926-e5544d26f373-tls-certs\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:14:08.755384 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.755345    2564 generic.go:356] "Generic (PLEG): container finished" podID="acb8e21f-a098-4e39-9926-e5544d26f373" containerID="284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b" exitCode=0
Jul 20 16:14:08.755872 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.755389    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"
Jul 20 16:14:08.755872 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.755395    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl" event={"ID":"acb8e21f-a098-4e39-9926-e5544d26f373","Type":"ContainerDied","Data":"284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b"}
Jul 20 16:14:08.755872 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.755436    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl" event={"ID":"acb8e21f-a098-4e39-9926-e5544d26f373","Type":"ContainerDied","Data":"071ee793366b4e6e2e4ba25eb1359547c9e392a66df4a5ad90206b6896d24424"}
Jul 20 16:14:08.755872 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.755458    2564 scope.go:117] "RemoveContainer" containerID="284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b"
Jul 20 16:14:08.764499 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.764481    2564 scope.go:117] "RemoveContainer" containerID="284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b"
Jul 20 16:14:08.764854 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:14:08.764833    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b\": container with ID starting with 284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b not found: ID does not exist" containerID="284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b"
Jul 20 16:14:08.764940 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.764866    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b"} err="failed to get container status \"284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b\": rpc error: code = NotFound desc = could not find container \"284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b\": container with ID starting with 284cc918fb402699d17661dbb9f55c5cb8df7a29dcc6dabcc0b419e7f71b7d5b not found: ID does not exist"
Jul 20 16:14:08.777385 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.777357    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"]
Jul 20 16:14:08.782015 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:08.781987    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-5d2e68d4-kserve-5d77c4d78c-52lrl"]
Jul 20 16:14:09.868440 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:09.868401    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acb8e21f-a098-4e39-9926-e5544d26f373" path="/var/lib/kubelet/pods/acb8e21f-a098-4e39-9926-e5544d26f373/volumes"
Jul 20 16:14:56.201591 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:14:56.201555    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:15:00.149948 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.149909    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742735-wgtgw"]
Jul 20 16:15:00.150431 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.150369    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acb8e21f-a098-4e39-9926-e5544d26f373" containerName="main"
Jul 20 16:15:00.150431 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.150388    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="acb8e21f-a098-4e39-9926-e5544d26f373" containerName="main"
Jul 20 16:15:00.150546 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.150470    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="acb8e21f-a098-4e39-9926-e5544d26f373" containerName="main"
Jul 20 16:15:00.154517 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.154494    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742735-wgtgw"
Jul 20 16:15:00.158177 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.158154    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:15:00.163608 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.163589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742735-wgtgw"
Jul 20 16:15:00.169122 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.169096    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742735-wgtgw"]
Jul 20 16:15:00.308637 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.308601    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742735-wgtgw"]
Jul 20 16:15:00.312382 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:15:00.312353    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd531f4_33fd_4fcf_8468_640d2bb56578.slice/crio-bcf8b2cfeb43748d8495d0fb96fcfb0faed88a4adadf321f19e442dfb845c6af WatchSource:0}: Error finding container bcf8b2cfeb43748d8495d0fb96fcfb0faed88a4adadf321f19e442dfb845c6af: Status 404 returned error can't find the container with id bcf8b2cfeb43748d8495d0fb96fcfb0faed88a4adadf321f19e442dfb845c6af
Jul 20 16:15:00.314194 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.314178    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:15:00.978676 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:00.978637    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742735-wgtgw" event={"ID":"6fd531f4-33fd-4fcf-8468-640d2bb56578","Type":"ContainerStarted","Data":"bcf8b2cfeb43748d8495d0fb96fcfb0faed88a4adadf321f19e442dfb845c6af"}
Jul 20 16:15:04.997964 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:04.997927    2564 generic.go:356] "Generic (PLEG): container finished" podID="6fd531f4-33fd-4fcf-8468-640d2bb56578" containerID="bb97c06c2e5c127b42729af0e501d63c87486f28b8763901fb394cf01997a4ac" exitCode=0
Jul 20 16:15:04.998312 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:04.997980    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742735-wgtgw" event={"ID":"6fd531f4-33fd-4fcf-8468-640d2bb56578","Type":"ContainerDied","Data":"bb97c06c2e5c127b42729af0e501d63c87486f28b8763901fb394cf01997a4ac"}
Jul 20 16:15:06.131308 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:06.131280    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742735-wgtgw"
Jul 20 16:15:07.009394 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:07.009361    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742735-wgtgw"
Jul 20 16:15:07.009394 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:07.009388    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742735-wgtgw" event={"ID":"6fd531f4-33fd-4fcf-8468-640d2bb56578","Type":"ContainerDied","Data":"bcf8b2cfeb43748d8495d0fb96fcfb0faed88a4adadf321f19e442dfb845c6af"}
Jul 20 16:15:07.009615 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:07.009437    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcf8b2cfeb43748d8495d0fb96fcfb0faed88a4adadf321f19e442dfb845c6af"
Jul 20 16:15:30.906965 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:30.906929    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:15:31.008945 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:31.008896    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:15:35.862158 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:35.862129    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:15:35.862654 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:15:35.862343    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:17:09.285243 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:09.285206    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:12.685169 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:12.685123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:20.617974 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:20.617941    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:26.151501 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:26.151463    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:33.380077 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:33.379997    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:38.812291 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:38.812254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:41.851641 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:41.851575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:41.947077 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:41.947042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:42.352948 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:42.352913    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:48.295697 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:48.295651    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:48.367082 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:48.367048    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:48.547562 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:48.547464    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:48.842108 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:48.842000    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:49.378825 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:49.378792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:50.770343 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:50.770302    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:50.871675 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:50.871638    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:50.916388 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:50.916355    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"]
Jul 20 16:17:50.916842 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:50.916827    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd531f4-33fd-4fcf-8468-640d2bb56578" containerName="cleanup"
Jul 20 16:17:50.916896 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:50.916846    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd531f4-33fd-4fcf-8468-640d2bb56578" containerName="cleanup"
Jul 20 16:17:50.916936 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:50.916922    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd531f4-33fd-4fcf-8468-640d2bb56578" containerName="cleanup"
Jul 20 16:17:50.920140 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:50.920119    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:50.922521 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:50.922500    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-eaab05e4-serving-cert\""
Jul 20 16:17:50.946636 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:50.946606    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"]
Jul 20 16:17:50.987122 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:50.987091    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:51.076889 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.076794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c35237e9-2f3b-47fc-84e1-8282d1347c19-maas-api-tls\") pod \"maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.076889 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.076856    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljzh7\" (UniqueName: \"kubernetes.io/projected/c35237e9-2f3b-47fc-84e1-8282d1347c19-kube-api-access-ljzh7\") pod \"maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.077077 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.076933    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c35237e9-2f3b-47fc-84e1-8282d1347c19-openshift-service-ca\") pod \"maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.178403 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.178367    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljzh7\" (UniqueName: \"kubernetes.io/projected/c35237e9-2f3b-47fc-84e1-8282d1347c19-kube-api-access-ljzh7\") pod \"maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.178403 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.178418    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c35237e9-2f3b-47fc-84e1-8282d1347c19-openshift-service-ca\") pod \"maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.178630 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.178463    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c35237e9-2f3b-47fc-84e1-8282d1347c19-maas-api-tls\") pod \"maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.178630 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:17:51.178555    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-serving-cert: secret "maas-api-e2e-ait-eaab05e4-serving-cert" not found
Jul 20 16:17:51.178630 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:17:51.178629    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c35237e9-2f3b-47fc-84e1-8282d1347c19-maas-api-tls podName:c35237e9-2f3b-47fc-84e1-8282d1347c19 nodeName:}" failed. No retries permitted until 2026-07-20 16:17:51.678606929 +0000 UTC m=+1636.333870184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c35237e9-2f3b-47fc-84e1-8282d1347c19-maas-api-tls") pod "maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft" (UID: "c35237e9-2f3b-47fc-84e1-8282d1347c19") : secret "maas-api-e2e-ait-eaab05e4-serving-cert" not found
Jul 20 16:17:51.179239 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.179217    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c35237e9-2f3b-47fc-84e1-8282d1347c19-openshift-service-ca\") pod \"maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.188643 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.188607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljzh7\" (UniqueName: \"kubernetes.io/projected/c35237e9-2f3b-47fc-84e1-8282d1347c19-kube-api-access-ljzh7\") pod \"maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.615275 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.615239    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:51.683179 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.683134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c35237e9-2f3b-47fc-84e1-8282d1347c19-maas-api-tls\") pod \"maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.685703 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.685666    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c35237e9-2f3b-47fc-84e1-8282d1347c19-maas-api-tls\") pod \"maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.709715 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.709654    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:17:51.829544 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.829514    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:51.976874 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:51.976714    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"]
Jul 20 16:17:51.979801 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:17:51.979766    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc35237e9_2f3b_47fc_84e1_8282d1347c19.slice/crio-b54b6affa889d8bddf2d78398fc877f8ebf505c1cd572e666eed5f02f41238b3 WatchSource:0}: Error finding container b54b6affa889d8bddf2d78398fc877f8ebf505c1cd572e666eed5f02f41238b3: Status 404 returned error can't find the container with id b54b6affa889d8bddf2d78398fc877f8ebf505c1cd572e666eed5f02f41238b3
Jul 20 16:17:52.704815 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:52.704769    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft" event={"ID":"c35237e9-2f3b-47fc-84e1-8282d1347c19","Type":"ContainerStarted","Data":"d6c7b56ff033348d3680925430bc88c8bcba1d00c32b9b29c3064526ae87aeff"}
Jul 20 16:17:52.705017 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:52.704861    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft" event={"ID":"c35237e9-2f3b-47fc-84e1-8282d1347c19","Type":"ContainerStarted","Data":"b54b6affa889d8bddf2d78398fc877f8ebf505c1cd572e666eed5f02f41238b3"}
Jul 20 16:17:52.705017 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:52.704901    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:17:52.725138 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:52.725084    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft" podStartSLOduration=2.127235691 podStartE2EDuration="2.72506678s" totalImagesPullingTime="597.831089ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:17:52.723462165 +0000 UTC m=+1637.378725450" watchObservedRunningTime="2026-07-20 16:17:52.72506678 +0000 UTC m=+1637.380330055"
Jul 20 16:17:58.714673 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:17:58.714640    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:18:16.284296 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.284262    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:16.298834 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.298801    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"]
Jul 20 16:18:16.302183 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.302160    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"
Jul 20 16:18:16.304158 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.304137    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:18:16.309397 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.309369    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"]
Jul 20 16:18:16.397064 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.397023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6048d685-4fa6-4351-ba27-9ea1d622879f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg\" (UID: \"6048d685-4fa6-4351-ba27-9ea1d622879f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"
Jul 20 16:18:16.498320 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.498282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6048d685-4fa6-4351-ba27-9ea1d622879f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg\" (UID: \"6048d685-4fa6-4351-ba27-9ea1d622879f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"
Jul 20 16:18:16.498978 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.498957    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6048d685-4fa6-4351-ba27-9ea1d622879f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg\" (UID: \"6048d685-4fa6-4351-ba27-9ea1d622879f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"
Jul 20 16:18:16.615095 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.615007    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"
Jul 20 16:18:16.753517 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.753486    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"]
Jul 20 16:18:16.757222 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:18:16.757180    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6048d685_4fa6_4351_ba27_9ea1d622879f.slice/crio-57e9ed9e32ecd69494697761070cda619d9b53522111b54848f55738054852ea WatchSource:0}: Error finding container 57e9ed9e32ecd69494697761070cda619d9b53522111b54848f55738054852ea: Status 404 returned error can't find the container with id 57e9ed9e32ecd69494697761070cda619d9b53522111b54848f55738054852ea
Jul 20 16:18:16.811676 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:16.811634    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg" event={"ID":"6048d685-4fa6-4351-ba27-9ea1d622879f","Type":"ContainerStarted","Data":"57e9ed9e32ecd69494697761070cda619d9b53522111b54848f55738054852ea"}
Jul 20 16:18:17.817293 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:17.817258    2564 generic.go:356] "Generic (PLEG): container finished" podID="6048d685-4fa6-4351-ba27-9ea1d622879f" containerID="2f329cde3cc255a9aaf2c1f7785e6b4ed4549a9e6c18921f2453fa59b90731e4" exitCode=0
Jul 20 16:18:17.817744 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:17.817356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg" event={"ID":"6048d685-4fa6-4351-ba27-9ea1d622879f","Type":"ContainerDied","Data":"2f329cde3cc255a9aaf2c1f7785e6b4ed4549a9e6c18921f2453fa59b90731e4"}
Jul 20 16:18:18.951264 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:18.951235    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"
Jul 20 16:18:19.017907 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:19.017867    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6048d685-4fa6-4351-ba27-9ea1d622879f-maas-api-service-ca\") pod \"6048d685-4fa6-4351-ba27-9ea1d622879f\" (UID: \"6048d685-4fa6-4351-ba27-9ea1d622879f\") "
Jul 20 16:18:19.018219 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:19.018196    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6048d685-4fa6-4351-ba27-9ea1d622879f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6048d685-4fa6-4351-ba27-9ea1d622879f" (UID: "6048d685-4fa6-4351-ba27-9ea1d622879f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:18:19.119193 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:19.119104    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6048d685-4fa6-4351-ba27-9ea1d622879f-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:18:19.827223 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:19.827190    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg" event={"ID":"6048d685-4fa6-4351-ba27-9ea1d622879f","Type":"ContainerDied","Data":"57e9ed9e32ecd69494697761070cda619d9b53522111b54848f55738054852ea"}
Jul 20 16:18:19.827406 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:19.827237    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57e9ed9e32ecd69494697761070cda619d9b53522111b54848f55738054852ea"
Jul 20 16:18:19.827406 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:19.827200    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"
Jul 20 16:18:26.721046 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:26.721006    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"]
Jul 20 16:18:26.721534 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:26.721262    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft" podUID="c35237e9-2f3b-47fc-84e1-8282d1347c19" containerName="maas-api" containerID="cri-o://d6c7b56ff033348d3680925430bc88c8bcba1d00c32b9b29c3064526ae87aeff" gracePeriod=30
Jul 20 16:18:26.740329 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:26.740300    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:26.864778 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:26.864741    2564 generic.go:356] "Generic (PLEG): container finished" podID="c35237e9-2f3b-47fc-84e1-8282d1347c19" containerID="d6c7b56ff033348d3680925430bc88c8bcba1d00c32b9b29c3064526ae87aeff" exitCode=0
Jul 20 16:18:26.864963 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:26.864809    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft" event={"ID":"c35237e9-2f3b-47fc-84e1-8282d1347c19","Type":"ContainerDied","Data":"d6c7b56ff033348d3680925430bc88c8bcba1d00c32b9b29c3064526ae87aeff"}
Jul 20 16:18:26.975188 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:26.975113    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:18:27.092909 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.092876    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c35237e9-2f3b-47fc-84e1-8282d1347c19-openshift-service-ca\") pod \"c35237e9-2f3b-47fc-84e1-8282d1347c19\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") "
Jul 20 16:18:27.093107 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.092950    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljzh7\" (UniqueName: \"kubernetes.io/projected/c35237e9-2f3b-47fc-84e1-8282d1347c19-kube-api-access-ljzh7\") pod \"c35237e9-2f3b-47fc-84e1-8282d1347c19\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") "
Jul 20 16:18:27.093107 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.093030    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c35237e9-2f3b-47fc-84e1-8282d1347c19-maas-api-tls\") pod \"c35237e9-2f3b-47fc-84e1-8282d1347c19\" (UID: \"c35237e9-2f3b-47fc-84e1-8282d1347c19\") "
Jul 20 16:18:27.093312 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.093286    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c35237e9-2f3b-47fc-84e1-8282d1347c19-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c35237e9-2f3b-47fc-84e1-8282d1347c19" (UID: "c35237e9-2f3b-47fc-84e1-8282d1347c19"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:18:27.095231 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.095203    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c35237e9-2f3b-47fc-84e1-8282d1347c19-kube-api-access-ljzh7" (OuterVolumeSpecName: "kube-api-access-ljzh7") pod "c35237e9-2f3b-47fc-84e1-8282d1347c19" (UID: "c35237e9-2f3b-47fc-84e1-8282d1347c19"). InnerVolumeSpecName "kube-api-access-ljzh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:18:27.095361 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.095265    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c35237e9-2f3b-47fc-84e1-8282d1347c19-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c35237e9-2f3b-47fc-84e1-8282d1347c19" (UID: "c35237e9-2f3b-47fc-84e1-8282d1347c19"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:18:27.193707 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.193646    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c35237e9-2f3b-47fc-84e1-8282d1347c19-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:18:27.193707 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.193677    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c35237e9-2f3b-47fc-84e1-8282d1347c19-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:18:27.193707 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.193712    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljzh7\" (UniqueName: \"kubernetes.io/projected/c35237e9-2f3b-47fc-84e1-8282d1347c19-kube-api-access-ljzh7\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:18:27.876288 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.876244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft" event={"ID":"c35237e9-2f3b-47fc-84e1-8282d1347c19","Type":"ContainerDied","Data":"b54b6affa889d8bddf2d78398fc877f8ebf505c1cd572e666eed5f02f41238b3"}
Jul 20 16:18:27.876738 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.876314    2564 scope.go:117] "RemoveContainer" containerID="d6c7b56ff033348d3680925430bc88c8bcba1d00c32b9b29c3064526ae87aeff"
Jul 20 16:18:27.876738 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.876269    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"
Jul 20 16:18:27.899327 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.899297    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"]
Jul 20 16:18:27.903896 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:27.903867    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaab05e4-6d77b9846c-w9wft"]
Jul 20 16:18:29.868973 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:29.868938    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c35237e9-2f3b-47fc-84e1-8282d1347c19" path="/var/lib/kubelet/pods/c35237e9-2f3b-47fc-84e1-8282d1347c19/volumes"
Jul 20 16:18:31.346768 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:31.346735    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:31.379464 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:31.379424    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"]
Jul 20 16:18:31.382708 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:31.382655    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaab05e4-pmbjg"]
Jul 20 16:18:31.826967 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:31.826937    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:31.868840 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:31.868807    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6048d685-4fa6-4351-ba27-9ea1d622879f" path="/var/lib/kubelet/pods/6048d685-4fa6-4351-ba27-9ea1d622879f/volumes"
Jul 20 16:18:31.879674 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:31.879634    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:32.012979 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:32.012946    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:32.111564 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:32.111477    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:32.220090 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:32.220059    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:32.536819 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:32.536785    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:32.902950 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:32.902844    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:35.093342 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.093310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:35.254706 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.254665    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"]
Jul 20 16:18:35.255025 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.255012    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c35237e9-2f3b-47fc-84e1-8282d1347c19" containerName="maas-api"
Jul 20 16:18:35.255079 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.255027    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c35237e9-2f3b-47fc-84e1-8282d1347c19" containerName="maas-api"
Jul 20 16:18:35.255079 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.255042    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6048d685-4fa6-4351-ba27-9ea1d622879f" containerName="revoke-keys"
Jul 20 16:18:35.255079 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.255048    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6048d685-4fa6-4351-ba27-9ea1d622879f" containerName="revoke-keys"
Jul 20 16:18:35.255177 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.255097    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c35237e9-2f3b-47fc-84e1-8282d1347c19" containerName="maas-api"
Jul 20 16:18:35.255177 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.255109    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6048d685-4fa6-4351-ba27-9ea1d622879f" containerName="revoke-keys"
Jul 20 16:18:35.258328 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.258303    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:35.261471 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.261446    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e0afadbb-serving-cert\""
Jul 20 16:18:35.270883 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.270856    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"]
Jul 20 16:18:35.278542 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.278511    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:35.363150 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.363061    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc7x5\" (UniqueName: \"kubernetes.io/projected/00f5027a-bd11-4451-a884-4cc1808bdaf1-kube-api-access-wc7x5\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:35.363150 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.363114    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f5027a-bd11-4451-a884-4cc1808bdaf1-openshift-service-ca\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:35.363333 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.363197    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:35.464436 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.464389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc7x5\" (UniqueName: \"kubernetes.io/projected/00f5027a-bd11-4451-a884-4cc1808bdaf1-kube-api-access-wc7x5\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:35.464641 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.464470    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f5027a-bd11-4451-a884-4cc1808bdaf1-openshift-service-ca\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:35.464641 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.464538    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:35.464807 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:18:35.464659    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-serving-cert: secret "maas-api-e2e-migrate-e0afadbb-serving-cert" not found
Jul 20 16:18:35.464807 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:18:35.464768    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls podName:00f5027a-bd11-4451-a884-4cc1808bdaf1 nodeName:}" failed. No retries permitted until 2026-07-20 16:18:35.964747529 +0000 UTC m=+1680.620010783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls") pod "maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd" (UID: "00f5027a-bd11-4451-a884-4cc1808bdaf1") : secret "maas-api-e2e-migrate-e0afadbb-serving-cert" not found
Jul 20 16:18:35.465233 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.465208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f5027a-bd11-4451-a884-4cc1808bdaf1-openshift-service-ca\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:35.478665 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.478631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc7x5\" (UniqueName: \"kubernetes.io/projected/00f5027a-bd11-4451-a884-4cc1808bdaf1-kube-api-access-wc7x5\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:35.969647 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:35.969612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:35.969832 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:18:35.969741    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-serving-cert: secret "maas-api-e2e-migrate-e0afadbb-serving-cert" not found
Jul 20 16:18:35.969832 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:18:35.969822    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls podName:00f5027a-bd11-4451-a884-4cc1808bdaf1 nodeName:}" failed. No retries permitted until 2026-07-20 16:18:36.969801402 +0000 UTC m=+1681.625064679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls") pod "maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd" (UID: "00f5027a-bd11-4451-a884-4cc1808bdaf1") : secret "maas-api-e2e-migrate-e0afadbb-serving-cert" not found
Jul 20 16:18:36.050933 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:36.050895    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:36.534969 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:36.534934    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:18:36.979035 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:36.978926    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:36.981437 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:36.981402    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls\") pod \"maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:37.068301 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:37.068252    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:37.206893 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:37.206867    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"]
Jul 20 16:18:37.209366 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:18:37.209335    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f5027a_bd11_4451_a884_4cc1808bdaf1.slice/crio-6f0bda75231903e1e62065dec340a695ed6a562f09bde30da04c2b1c8b4c5559 WatchSource:0}: Error finding container 6f0bda75231903e1e62065dec340a695ed6a562f09bde30da04c2b1c8b4c5559: Status 404 returned error can't find the container with id 6f0bda75231903e1e62065dec340a695ed6a562f09bde30da04c2b1c8b4c5559
Jul 20 16:18:37.921344 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:37.921256    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd" event={"ID":"00f5027a-bd11-4451-a884-4cc1808bdaf1","Type":"ContainerStarted","Data":"90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23"}
Jul 20 16:18:37.921344 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:37.921305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd" event={"ID":"00f5027a-bd11-4451-a884-4cc1808bdaf1","Type":"ContainerStarted","Data":"6f0bda75231903e1e62065dec340a695ed6a562f09bde30da04c2b1c8b4c5559"}
Jul 20 16:18:37.921763 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:37.921382    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:18:37.938907 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:37.938852    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd" podStartSLOduration=2.597462674 podStartE2EDuration="2.938836506s" totalImagesPullingTime="341.373832ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:18:37.937234957 +0000 UTC m=+1682.592498235" watchObservedRunningTime="2026-07-20 16:18:37.938836506 +0000 UTC m=+1682.594099782"
Jul 20 16:18:43.931283 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:18:43.931249    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:19:00.077060 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:00.077023    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"]
Jul 20 16:19:00.080340 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:00.080303    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"
Jul 20 16:19:00.082454 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:00.082431    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:19:00.088185 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:00.087539    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"]
Jul 20 16:19:00.179164 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:00.179122    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f759cd-8e83-473c-8d08-8b35228c0f70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt\" (UID: \"45f759cd-8e83-473c-8d08-8b35228c0f70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"
Jul 20 16:19:00.280118 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:00.280085    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f759cd-8e83-473c-8d08-8b35228c0f70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt\" (UID: \"45f759cd-8e83-473c-8d08-8b35228c0f70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"
Jul 20 16:19:00.280762 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:00.280740    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f759cd-8e83-473c-8d08-8b35228c0f70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt\" (UID: \"45f759cd-8e83-473c-8d08-8b35228c0f70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"
Jul 20 16:19:00.392232 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:00.392148    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"
Jul 20 16:19:00.530099 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:00.530065    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"]
Jul 20 16:19:00.532342 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:19:00.532303    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45f759cd_8e83_473c_8d08_8b35228c0f70.slice/crio-c1b6c8e52e7e8bdf3523a9640ba9df4d122c9842bd19758f1ef7acb4a74c0892 WatchSource:0}: Error finding container c1b6c8e52e7e8bdf3523a9640ba9df4d122c9842bd19758f1ef7acb4a74c0892: Status 404 returned error can't find the container with id c1b6c8e52e7e8bdf3523a9640ba9df4d122c9842bd19758f1ef7acb4a74c0892
Jul 20 16:19:01.026867 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:01.026824    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt" event={"ID":"45f759cd-8e83-473c-8d08-8b35228c0f70","Type":"ContainerStarted","Data":"0e39d6115e984e16b6fb5fab7e92f4378464d8a78d3cae992707a6d3122a3c74"}
Jul 20 16:19:01.027072 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:01.026888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt" event={"ID":"45f759cd-8e83-473c-8d08-8b35228c0f70","Type":"ContainerStarted","Data":"c1b6c8e52e7e8bdf3523a9640ba9df4d122c9842bd19758f1ef7acb4a74c0892"}
Jul 20 16:19:01.043635 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:01.043583    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt" podStartSLOduration=1.043567335 podStartE2EDuration="1.043567335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:19:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:19:01.041965931 +0000 UTC m=+1705.697229199" watchObservedRunningTime="2026-07-20 16:19:01.043567335 +0000 UTC m=+1705.698830611"
Jul 20 16:19:02.032649 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:02.032617    2564 generic.go:356] "Generic (PLEG): container finished" podID="45f759cd-8e83-473c-8d08-8b35228c0f70" containerID="0e39d6115e984e16b6fb5fab7e92f4378464d8a78d3cae992707a6d3122a3c74" exitCode=0
Jul 20 16:19:02.033064 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:02.032717    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt" event={"ID":"45f759cd-8e83-473c-8d08-8b35228c0f70","Type":"ContainerDied","Data":"0e39d6115e984e16b6fb5fab7e92f4378464d8a78d3cae992707a6d3122a3c74"}
Jul 20 16:19:03.171866 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:03.171832    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"
Jul 20 16:19:03.206121 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:03.206089    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f759cd-8e83-473c-8d08-8b35228c0f70-maas-api-service-ca\") pod \"45f759cd-8e83-473c-8d08-8b35228c0f70\" (UID: \"45f759cd-8e83-473c-8d08-8b35228c0f70\") "
Jul 20 16:19:03.206460 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:03.206428    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45f759cd-8e83-473c-8d08-8b35228c0f70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "45f759cd-8e83-473c-8d08-8b35228c0f70" (UID: "45f759cd-8e83-473c-8d08-8b35228c0f70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:19:03.307475 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:03.307383    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f759cd-8e83-473c-8d08-8b35228c0f70-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:19:04.042314 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:04.042268    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt" event={"ID":"45f759cd-8e83-473c-8d08-8b35228c0f70","Type":"ContainerDied","Data":"c1b6c8e52e7e8bdf3523a9640ba9df4d122c9842bd19758f1ef7acb4a74c0892"}
Jul 20 16:19:04.042459 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:04.042324    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1b6c8e52e7e8bdf3523a9640ba9df4d122c9842bd19758f1ef7acb4a74c0892"
Jul 20 16:19:04.042459 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:04.042281    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"
Jul 20 16:19:10.717237 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:10.717198    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"]
Jul 20 16:19:10.717712 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:10.717546    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd" podUID="00f5027a-bd11-4451-a884-4cc1808bdaf1" containerName="maas-api" containerID="cri-o://90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23" gracePeriod=30
Jul 20 16:19:10.739960 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:10.739928    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:10.971365 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:10.971293    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:19:11.071933 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.071900    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls\") pod \"00f5027a-bd11-4451-a884-4cc1808bdaf1\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") "
Jul 20 16:19:11.072081 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.072013    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc7x5\" (UniqueName: \"kubernetes.io/projected/00f5027a-bd11-4451-a884-4cc1808bdaf1-kube-api-access-wc7x5\") pod \"00f5027a-bd11-4451-a884-4cc1808bdaf1\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") "
Jul 20 16:19:11.072081 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.072040    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f5027a-bd11-4451-a884-4cc1808bdaf1-openshift-service-ca\") pod \"00f5027a-bd11-4451-a884-4cc1808bdaf1\" (UID: \"00f5027a-bd11-4451-a884-4cc1808bdaf1\") "
Jul 20 16:19:11.072445 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.072416    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f5027a-bd11-4451-a884-4cc1808bdaf1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "00f5027a-bd11-4451-a884-4cc1808bdaf1" (UID: "00f5027a-bd11-4451-a884-4cc1808bdaf1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:19:11.074341 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.074313    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00f5027a-bd11-4451-a884-4cc1808bdaf1-kube-api-access-wc7x5" (OuterVolumeSpecName: "kube-api-access-wc7x5") pod "00f5027a-bd11-4451-a884-4cc1808bdaf1" (UID: "00f5027a-bd11-4451-a884-4cc1808bdaf1"). InnerVolumeSpecName "kube-api-access-wc7x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:19:11.074448 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.074336    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "00f5027a-bd11-4451-a884-4cc1808bdaf1" (UID: "00f5027a-bd11-4451-a884-4cc1808bdaf1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:19:11.075498 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.075474    2564 generic.go:356] "Generic (PLEG): container finished" podID="00f5027a-bd11-4451-a884-4cc1808bdaf1" containerID="90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23" exitCode=0
Jul 20 16:19:11.075588 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.075509    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd" event={"ID":"00f5027a-bd11-4451-a884-4cc1808bdaf1","Type":"ContainerDied","Data":"90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23"}
Jul 20 16:19:11.075588 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.075556    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd" event={"ID":"00f5027a-bd11-4451-a884-4cc1808bdaf1","Type":"ContainerDied","Data":"6f0bda75231903e1e62065dec340a695ed6a562f09bde30da04c2b1c8b4c5559"}
Jul 20 16:19:11.075588 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.075572    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"
Jul 20 16:19:11.075588 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.075579    2564 scope.go:117] "RemoveContainer" containerID="90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23"
Jul 20 16:19:11.085504 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.085486    2564 scope.go:117] "RemoveContainer" containerID="90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23"
Jul 20 16:19:11.085902 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:19:11.085882    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23\": container with ID starting with 90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23 not found: ID does not exist" containerID="90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23"
Jul 20 16:19:11.085976 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.085911    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23"} err="failed to get container status \"90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23\": rpc error: code = NotFound desc = could not find container \"90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23\": container with ID starting with 90bfad8cb3b91afb293e245d565f11f9a55b3bf4e6b136c1b47078405a2e1a23 not found: ID does not exist"
Jul 20 16:19:11.098227 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.098195    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"]
Jul 20 16:19:11.106038 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.103508    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0afadbb-6bbddcbd5b-flncd"]
Jul 20 16:19:11.173471 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.173433    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc7x5\" (UniqueName: \"kubernetes.io/projected/00f5027a-bd11-4451-a884-4cc1808bdaf1-kube-api-access-wc7x5\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:19:11.173471 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.173464    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f5027a-bd11-4451-a884-4cc1808bdaf1-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:19:11.173471 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.173475    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f5027a-bd11-4451-a884-4cc1808bdaf1-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:19:11.869446 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:11.869395    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00f5027a-bd11-4451-a884-4cc1808bdaf1" path="/var/lib/kubelet/pods/00f5027a-bd11-4451-a884-4cc1808bdaf1/volumes"
Jul 20 16:19:15.165087 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:15.165051    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:15.170180 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:15.170152    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"]
Jul 20 16:19:15.172511 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:15.172486    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e0afadbb-zb5bt"]
Jul 20 16:19:15.316773 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:15.316742    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:15.559584 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:15.559551    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:15.777802 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:15.777771    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:15.868923 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:15.868848    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45f759cd-8e83-473c-8d08-8b35228c0f70" path="/var/lib/kubelet/pods/45f759cd-8e83-473c-8d08-8b35228c0f70/volumes"
Jul 20 16:19:16.087566 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:16.087535    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:16.373543 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:16.373515    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:18.235885 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.235851    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:18.339536 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.339502    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:18.354909 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.354878    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"]
Jul 20 16:19:18.355226 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.355213    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45f759cd-8e83-473c-8d08-8b35228c0f70" containerName="revoke-keys"
Jul 20 16:19:18.355226 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.355227    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="45f759cd-8e83-473c-8d08-8b35228c0f70" containerName="revoke-keys"
Jul 20 16:19:18.355307 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.355239    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f5027a-bd11-4451-a884-4cc1808bdaf1" containerName="maas-api"
Jul 20 16:19:18.355307 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.355245    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f5027a-bd11-4451-a884-4cc1808bdaf1" containerName="maas-api"
Jul 20 16:19:18.355307 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.355300    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="45f759cd-8e83-473c-8d08-8b35228c0f70" containerName="revoke-keys"
Jul 20 16:19:18.355307 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.355307    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f5027a-bd11-4451-a884-4cc1808bdaf1" containerName="maas-api"
Jul 20 16:19:18.359371 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.359350    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:18.363131 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.363107    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-babce7f9-serving-cert\""
Jul 20 16:19:18.371915 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.371884    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"]
Jul 20 16:19:18.435350 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.435314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:18.443393 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.443362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726193fa-263b-4095-b474-0612de4cb410-maas-api-tls\") pod \"maas-api-e2e-ait-babce7f9-848fcb4559-27bgw\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:18.443563 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.443492    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lblrn\" (UniqueName: \"kubernetes.io/projected/726193fa-263b-4095-b474-0612de4cb410-kube-api-access-lblrn\") pod \"maas-api-e2e-ait-babce7f9-848fcb4559-27bgw\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:18.443626 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.443590    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/726193fa-263b-4095-b474-0612de4cb410-openshift-service-ca\") pod \"maas-api-e2e-ait-babce7f9-848fcb4559-27bgw\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:18.544784 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.544749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/726193fa-263b-4095-b474-0612de4cb410-openshift-service-ca\") pod \"maas-api-e2e-ait-babce7f9-848fcb4559-27bgw\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:18.544964 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.544799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726193fa-263b-4095-b474-0612de4cb410-maas-api-tls\") pod \"maas-api-e2e-ait-babce7f9-848fcb4559-27bgw\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:18.544964 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.544881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lblrn\" (UniqueName: \"kubernetes.io/projected/726193fa-263b-4095-b474-0612de4cb410-kube-api-access-lblrn\") pod \"maas-api-e2e-ait-babce7f9-848fcb4559-27bgw\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:18.545043 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:19:18.544969    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-serving-cert: secret "maas-api-e2e-ait-babce7f9-serving-cert" not found
Jul 20 16:19:18.545095 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:19:18.545081    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/726193fa-263b-4095-b474-0612de4cb410-maas-api-tls podName:726193fa-263b-4095-b474-0612de4cb410 nodeName:}" failed. No retries permitted until 2026-07-20 16:19:19.045057464 +0000 UTC m=+1723.700320719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/726193fa-263b-4095-b474-0612de4cb410-maas-api-tls") pod "maas-api-e2e-ait-babce7f9-848fcb4559-27bgw" (UID: "726193fa-263b-4095-b474-0612de4cb410") : secret "maas-api-e2e-ait-babce7f9-serving-cert" not found
Jul 20 16:19:18.545488 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.545469    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/726193fa-263b-4095-b474-0612de4cb410-openshift-service-ca\") pod \"maas-api-e2e-ait-babce7f9-848fcb4559-27bgw\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:18.559864 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.559834    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lblrn\" (UniqueName: \"kubernetes.io/projected/726193fa-263b-4095-b474-0612de4cb410-kube-api-access-lblrn\") pod \"maas-api-e2e-ait-babce7f9-848fcb4559-27bgw\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:18.639568 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.639536    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:18.947914 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:18.947824    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:19.047990 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:19.047961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:19.050123 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:19.050100    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726193fa-263b-4095-b474-0612de4cb410-maas-api-tls\") pod \"maas-api-e2e-ait-babce7f9-848fcb4559-27bgw\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:19.052593 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:19.052564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726193fa-263b-4095-b474-0612de4cb410-maas-api-tls\") pod \"maas-api-e2e-ait-babce7f9-848fcb4559-27bgw\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:19.269316 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:19.269286    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:19.414746 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:19.414721    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"]
Jul 20 16:19:19.417348 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:19:19.417313    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726193fa_263b_4095_b474_0612de4cb410.slice/crio-f8a093bbf23fab353e72e8873655fcb067ca91bc8dbc81bcc62359f080f5f31a WatchSource:0}: Error finding container f8a093bbf23fab353e72e8873655fcb067ca91bc8dbc81bcc62359f080f5f31a: Status 404 returned error can't find the container with id f8a093bbf23fab353e72e8873655fcb067ca91bc8dbc81bcc62359f080f5f31a
Jul 20 16:19:20.119992 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:20.119956    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw" event={"ID":"726193fa-263b-4095-b474-0612de4cb410","Type":"ContainerStarted","Data":"1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89"}
Jul 20 16:19:20.120150 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:20.120010    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw" event={"ID":"726193fa-263b-4095-b474-0612de4cb410","Type":"ContainerStarted","Data":"f8a093bbf23fab353e72e8873655fcb067ca91bc8dbc81bcc62359f080f5f31a"}
Jul 20 16:19:20.120150 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:20.120101    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:20.137010 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:20.136940    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw" podStartSLOduration=1.658500186 podStartE2EDuration="2.136921678s" totalImagesPullingTime="478.421492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:19:20.135888507 +0000 UTC m=+1724.791151787" watchObservedRunningTime="2026-07-20 16:19:20.136921678 +0000 UTC m=+1724.792184953"
Jul 20 16:19:26.130376 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:26.130346    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:44.007153 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:44.007107    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"]
Jul 20 16:19:44.010568 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:44.010541    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"
Jul 20 16:19:44.012671 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:44.012640    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:19:44.018809 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:44.018207    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"]
Jul 20 16:19:44.064553 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:44.064504    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda97fac-6c3f-432a-8423-e99cce459f05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-babce7f9-6fpft\" (UID: \"eda97fac-6c3f-432a-8423-e99cce459f05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"
Jul 20 16:19:44.164984 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:44.164949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda97fac-6c3f-432a-8423-e99cce459f05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-babce7f9-6fpft\" (UID: \"eda97fac-6c3f-432a-8423-e99cce459f05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"
Jul 20 16:19:44.165640 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:44.165623    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda97fac-6c3f-432a-8423-e99cce459f05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-babce7f9-6fpft\" (UID: \"eda97fac-6c3f-432a-8423-e99cce459f05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"
Jul 20 16:19:44.324292 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:44.324257    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"
Jul 20 16:19:44.463439 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:44.463402    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"]
Jul 20 16:19:44.466987 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:19:44.466949    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda97fac_6c3f_432a_8423_e99cce459f05.slice/crio-340b43a7d53e73800cdab501d7009af041d9b32e82b9c25ce6e54cd71ae1d070 WatchSource:0}: Error finding container 340b43a7d53e73800cdab501d7009af041d9b32e82b9c25ce6e54cd71ae1d070: Status 404 returned error can't find the container with id 340b43a7d53e73800cdab501d7009af041d9b32e82b9c25ce6e54cd71ae1d070
Jul 20 16:19:45.234762 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:45.234727    2564 generic.go:356] "Generic (PLEG): container finished" podID="eda97fac-6c3f-432a-8423-e99cce459f05" containerID="5ac16c1dc6220c71341cab2fc9ce43ae8cd5096c451e4173c1346b89e18be27a" exitCode=0
Jul 20 16:19:45.235172 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:45.234780    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft" event={"ID":"eda97fac-6c3f-432a-8423-e99cce459f05","Type":"ContainerDied","Data":"5ac16c1dc6220c71341cab2fc9ce43ae8cd5096c451e4173c1346b89e18be27a"}
Jul 20 16:19:45.235172 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:45.234817    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft" event={"ID":"eda97fac-6c3f-432a-8423-e99cce459f05","Type":"ContainerStarted","Data":"340b43a7d53e73800cdab501d7009af041d9b32e82b9c25ce6e54cd71ae1d070"}
Jul 20 16:19:46.373498 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:46.373473    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"
Jul 20 16:19:46.379920 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:46.379891    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda97fac-6c3f-432a-8423-e99cce459f05-maas-api-service-ca\") pod \"eda97fac-6c3f-432a-8423-e99cce459f05\" (UID: \"eda97fac-6c3f-432a-8423-e99cce459f05\") "
Jul 20 16:19:46.380211 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:46.380192    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eda97fac-6c3f-432a-8423-e99cce459f05-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eda97fac-6c3f-432a-8423-e99cce459f05" (UID: "eda97fac-6c3f-432a-8423-e99cce459f05"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:19:46.480610 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:46.480572    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda97fac-6c3f-432a-8423-e99cce459f05-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:19:47.040860 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:47.040825    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:47.136881 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:47.136851    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:47.244726 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:47.244678    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"
Jul 20 16:19:47.244726 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:47.244715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft" event={"ID":"eda97fac-6c3f-432a-8423-e99cce459f05","Type":"ContainerDied","Data":"340b43a7d53e73800cdab501d7009af041d9b32e82b9c25ce6e54cd71ae1d070"}
Jul 20 16:19:47.244944 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:47.244759    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="340b43a7d53e73800cdab501d7009af041d9b32e82b9c25ce6e54cd71ae1d070"
Jul 20 16:19:54.596844 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:54.596804    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:54.700669 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:54.700639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:54.839009 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:54.838961    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"]
Jul 20 16:19:54.839357 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:54.839300    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw" podUID="726193fa-263b-4095-b474-0612de4cb410" containerName="maas-api" containerID="cri-o://1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89" gracePeriod=30
Jul 20 16:19:54.889951 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:54.889909    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:55.091018 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.090987    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:55.141757 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.141642    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lblrn\" (UniqueName: \"kubernetes.io/projected/726193fa-263b-4095-b474-0612de4cb410-kube-api-access-lblrn\") pod \"726193fa-263b-4095-b474-0612de4cb410\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") "
Jul 20 16:19:55.141757 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.141717    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726193fa-263b-4095-b474-0612de4cb410-maas-api-tls\") pod \"726193fa-263b-4095-b474-0612de4cb410\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") "
Jul 20 16:19:55.141995 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.141818    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/726193fa-263b-4095-b474-0612de4cb410-openshift-service-ca\") pod \"726193fa-263b-4095-b474-0612de4cb410\" (UID: \"726193fa-263b-4095-b474-0612de4cb410\") "
Jul 20 16:19:55.142167 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.142137    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/726193fa-263b-4095-b474-0612de4cb410-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "726193fa-263b-4095-b474-0612de4cb410" (UID: "726193fa-263b-4095-b474-0612de4cb410"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:19:55.143792 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.143765    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/726193fa-263b-4095-b474-0612de4cb410-kube-api-access-lblrn" (OuterVolumeSpecName: "kube-api-access-lblrn") pod "726193fa-263b-4095-b474-0612de4cb410" (UID: "726193fa-263b-4095-b474-0612de4cb410"). InnerVolumeSpecName "kube-api-access-lblrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:19:55.143792 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.143777    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/726193fa-263b-4095-b474-0612de4cb410-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "726193fa-263b-4095-b474-0612de4cb410" (UID: "726193fa-263b-4095-b474-0612de4cb410"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:19:55.242719 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.242641    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lblrn\" (UniqueName: \"kubernetes.io/projected/726193fa-263b-4095-b474-0612de4cb410-kube-api-access-lblrn\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:19:55.242719 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.242717    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726193fa-263b-4095-b474-0612de4cb410-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:19:55.242719 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.242730    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/726193fa-263b-4095-b474-0612de4cb410-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:19:55.280579 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.280540    2564 generic.go:356] "Generic (PLEG): container finished" podID="726193fa-263b-4095-b474-0612de4cb410" containerID="1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89" exitCode=0
Jul 20 16:19:55.280754 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.280595    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"
Jul 20 16:19:55.280754 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.280618    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw" event={"ID":"726193fa-263b-4095-b474-0612de4cb410","Type":"ContainerDied","Data":"1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89"}
Jul 20 16:19:55.280754 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.280671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw" event={"ID":"726193fa-263b-4095-b474-0612de4cb410","Type":"ContainerDied","Data":"f8a093bbf23fab353e72e8873655fcb067ca91bc8dbc81bcc62359f080f5f31a"}
Jul 20 16:19:55.280754 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.280717    2564 scope.go:117] "RemoveContainer" containerID="1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89"
Jul 20 16:19:55.289667 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.289647    2564 scope.go:117] "RemoveContainer" containerID="1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89"
Jul 20 16:19:55.290018 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:19:55.289997    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89\": container with ID starting with 1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89 not found: ID does not exist" containerID="1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89"
Jul 20 16:19:55.290104 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.290031    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89"} err="failed to get container status \"1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89\": rpc error: code = NotFound desc = could not find container \"1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89\": container with ID starting with 1e8ceac8aad61fd41a0b50fb6a6a4b2ef8604f84e4de72999e179b7f56b5ee89 not found: ID does not exist"
Jul 20 16:19:55.300904 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.300877    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"]
Jul 20 16:19:55.306740 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.306716    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-babce7f9-848fcb4559-27bgw"]
Jul 20 16:19:55.871570 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:55.871527    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="726193fa-263b-4095-b474-0612de4cb410" path="/var/lib/kubelet/pods/726193fa-263b-4095-b474-0612de4cb410/volumes"
Jul 20 16:19:59.120241 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:59.120152    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"]
Jul 20 16:19:59.122378 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:59.122355    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-babce7f9-6fpft"]
Jul 20 16:19:59.127660 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:59.127633    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:19:59.868046 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:19:59.868007    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eda97fac-6c3f-432a-8423-e99cce459f05" path="/var/lib/kubelet/pods/eda97fac-6c3f-432a-8423-e99cce459f05/volumes"
Jul 20 16:20:00.311989 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:00.311956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:00.579314 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:00.579208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:00.805457 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:00.805428    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:01.196629 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:01.196588    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:01.666345 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:01.666310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:03.491002 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.490962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:03.572990 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.572953    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:03.581347 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.581313    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"]
Jul 20 16:20:03.581967 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.581945    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eda97fac-6c3f-432a-8423-e99cce459f05" containerName="revoke-keys"
Jul 20 16:20:03.581967 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.581968    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eda97fac-6c3f-432a-8423-e99cce459f05" containerName="revoke-keys"
Jul 20 16:20:03.582146 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.581997    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="726193fa-263b-4095-b474-0612de4cb410" containerName="maas-api"
Jul 20 16:20:03.582146 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.582006    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="726193fa-263b-4095-b474-0612de4cb410" containerName="maas-api"
Jul 20 16:20:03.582245 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.582192    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="726193fa-263b-4095-b474-0612de4cb410" containerName="maas-api"
Jul 20 16:20:03.582245 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.582206    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eda97fac-6c3f-432a-8423-e99cce459f05" containerName="revoke-keys"
Jul 20 16:20:03.587366 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.587336    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:03.589380 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.589359    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6679b04f-serving-cert\""
Jul 20 16:20:03.592721 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.592678    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"]
Jul 20 16:20:03.601111 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.601081    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b73bac-1643-4be9-85c3-68ccb2bd60c9-openshift-service-ca\") pod \"maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:03.601247 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.601124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b73bac-1643-4be9-85c3-68ccb2bd60c9-maas-api-tls\") pod \"maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:03.601289 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.601257    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr266\" (UniqueName: \"kubernetes.io/projected/07b73bac-1643-4be9-85c3-68ccb2bd60c9-kube-api-access-pr266\") pod \"maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:03.675430 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.675394    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:03.702310 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.702269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b73bac-1643-4be9-85c3-68ccb2bd60c9-openshift-service-ca\") pod \"maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:03.702502 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.702336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b73bac-1643-4be9-85c3-68ccb2bd60c9-maas-api-tls\") pod \"maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:03.702502 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.702431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr266\" (UniqueName: \"kubernetes.io/projected/07b73bac-1643-4be9-85c3-68ccb2bd60c9-kube-api-access-pr266\") pod \"maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:03.702502 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:20:03.702455    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-serving-cert: secret "maas-api-e2e-derive-6679b04f-serving-cert" not found
Jul 20 16:20:03.702609 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:20:03.702529    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07b73bac-1643-4be9-85c3-68ccb2bd60c9-maas-api-tls podName:07b73bac-1643-4be9-85c3-68ccb2bd60c9 nodeName:}" failed. No retries permitted until 2026-07-20 16:20:04.202506864 +0000 UTC m=+1768.857770117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07b73bac-1643-4be9-85c3-68ccb2bd60c9-maas-api-tls") pod "maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f" (UID: "07b73bac-1643-4be9-85c3-68ccb2bd60c9") : secret "maas-api-e2e-derive-6679b04f-serving-cert" not found
Jul 20 16:20:03.703074 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.703057    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b73bac-1643-4be9-85c3-68ccb2bd60c9-openshift-service-ca\") pod \"maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:03.712022 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.711986    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr266\" (UniqueName: \"kubernetes.io/projected/07b73bac-1643-4be9-85c3-68ccb2bd60c9-kube-api-access-pr266\") pod \"maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:03.867049 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:03.867009    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:04.207112 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:04.207022    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b73bac-1643-4be9-85c3-68ccb2bd60c9-maas-api-tls\") pod \"maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:04.209601 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:04.209564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b73bac-1643-4be9-85c3-68ccb2bd60c9-maas-api-tls\") pod \"maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:04.280480 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:04.280442    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:04.342209 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:04.342171    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:04.500152 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:04.500058    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:04.643607 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:04.643574    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"]
Jul 20 16:20:04.647017 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:20:04.646979    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b73bac_1643_4be9_85c3_68ccb2bd60c9.slice/crio-f53d22df5771f39a9363e1dc5b272e3582673ac841c4b77ff924dd075f993e77 WatchSource:0}: Error finding container f53d22df5771f39a9363e1dc5b272e3582673ac841c4b77ff924dd075f993e77: Status 404 returned error can't find the container with id f53d22df5771f39a9363e1dc5b272e3582673ac841c4b77ff924dd075f993e77
Jul 20 16:20:04.648441 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:04.648422    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:20:05.329825 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:05.329784    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f" event={"ID":"07b73bac-1643-4be9-85c3-68ccb2bd60c9","Type":"ContainerStarted","Data":"65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e"}
Jul 20 16:20:05.330015 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:05.329851    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:05.330015 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:05.329872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f" event={"ID":"07b73bac-1643-4be9-85c3-68ccb2bd60c9","Type":"ContainerStarted","Data":"f53d22df5771f39a9363e1dc5b272e3582673ac841c4b77ff924dd075f993e77"}
Jul 20 16:20:05.347088 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:05.347032    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f" podStartSLOduration=1.828880779 podStartE2EDuration="2.347016589s" totalImagesPullingTime="518.13581ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:20:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:20:05.345414755 +0000 UTC m=+1770.000678032" watchObservedRunningTime="2026-07-20 16:20:05.347016589 +0000 UTC m=+1770.002279865"
Jul 20 16:20:11.339741 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:11.339712    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:28.385304 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:28.385217    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"]
Jul 20 16:20:28.388674 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:28.388652    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"
Jul 20 16:20:28.391032 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:28.390999    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:20:28.397104 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:28.397076    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"]
Jul 20 16:20:28.517653 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:28.517612    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8e6597-26d6-4c10-a60b-6ef857f9478f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk\" (UID: \"9e8e6597-26d6-4c10-a60b-6ef857f9478f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"
Jul 20 16:20:28.619012 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:28.618970    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8e6597-26d6-4c10-a60b-6ef857f9478f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk\" (UID: \"9e8e6597-26d6-4c10-a60b-6ef857f9478f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"
Jul 20 16:20:28.619636 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:28.619614    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8e6597-26d6-4c10-a60b-6ef857f9478f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk\" (UID: \"9e8e6597-26d6-4c10-a60b-6ef857f9478f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"
Jul 20 16:20:28.699797 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:28.699679    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"
Jul 20 16:20:28.843922 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:28.843883    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"]
Jul 20 16:20:28.847987 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:20:28.847955    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8e6597_26d6_4c10_a60b_6ef857f9478f.slice/crio-c2d28c8bdb3823755133dbbc59b5d9bfeadc43973ccb4ce23beabefee56a47ca WatchSource:0}: Error finding container c2d28c8bdb3823755133dbbc59b5d9bfeadc43973ccb4ce23beabefee56a47ca: Status 404 returned error can't find the container with id c2d28c8bdb3823755133dbbc59b5d9bfeadc43973ccb4ce23beabefee56a47ca
Jul 20 16:20:29.439851 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:29.439754    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk" event={"ID":"9e8e6597-26d6-4c10-a60b-6ef857f9478f","Type":"ContainerStarted","Data":"4126da263642a955c48312f21a10e9f96e240323f50f4c04cfab12d1897781e1"}
Jul 20 16:20:29.439851 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:29.439818    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk" event={"ID":"9e8e6597-26d6-4c10-a60b-6ef857f9478f","Type":"ContainerStarted","Data":"c2d28c8bdb3823755133dbbc59b5d9bfeadc43973ccb4ce23beabefee56a47ca"}
Jul 20 16:20:29.456206 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:29.456154    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk" podStartSLOduration=1.456135934 podStartE2EDuration="1.456135934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:20:29.456061444 +0000 UTC m=+1794.111324720" watchObservedRunningTime="2026-07-20 16:20:29.456135934 +0000 UTC m=+1794.111399209"
Jul 20 16:20:30.445510 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:30.445477    2564 generic.go:356] "Generic (PLEG): container finished" podID="9e8e6597-26d6-4c10-a60b-6ef857f9478f" containerID="4126da263642a955c48312f21a10e9f96e240323f50f4c04cfab12d1897781e1" exitCode=0
Jul 20 16:20:30.445906 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:30.445534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk" event={"ID":"9e8e6597-26d6-4c10-a60b-6ef857f9478f","Type":"ContainerDied","Data":"4126da263642a955c48312f21a10e9f96e240323f50f4c04cfab12d1897781e1"}
Jul 20 16:20:31.581242 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:31.581210    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"
Jul 20 16:20:31.645908 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:31.645874    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8e6597-26d6-4c10-a60b-6ef857f9478f-maas-api-service-ca\") pod \"9e8e6597-26d6-4c10-a60b-6ef857f9478f\" (UID: \"9e8e6597-26d6-4c10-a60b-6ef857f9478f\") "
Jul 20 16:20:31.646218 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:31.646195    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8e6597-26d6-4c10-a60b-6ef857f9478f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e8e6597-26d6-4c10-a60b-6ef857f9478f" (UID: "9e8e6597-26d6-4c10-a60b-6ef857f9478f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:20:31.747278 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:31.747172    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8e6597-26d6-4c10-a60b-6ef857f9478f-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:20:32.455083 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:32.455045    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk" event={"ID":"9e8e6597-26d6-4c10-a60b-6ef857f9478f","Type":"ContainerDied","Data":"c2d28c8bdb3823755133dbbc59b5d9bfeadc43973ccb4ce23beabefee56a47ca"}
Jul 20 16:20:32.455259 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:32.455122    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2d28c8bdb3823755133dbbc59b5d9bfeadc43973ccb4ce23beabefee56a47ca"
Jul 20 16:20:32.455259 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:32.455053    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"
Jul 20 16:20:35.889063 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:35.889036    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:20:35.889555 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:35.889422    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:20:39.075615 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.075577    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"]
Jul 20 16:20:39.076072 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.075802    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f" podUID="07b73bac-1643-4be9-85c3-68ccb2bd60c9" containerName="maas-api" containerID="cri-o://65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e" gracePeriod=30
Jul 20 16:20:39.120323 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.120292    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:39.324281 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.324256    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:39.411075 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.410975    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr266\" (UniqueName: \"kubernetes.io/projected/07b73bac-1643-4be9-85c3-68ccb2bd60c9-kube-api-access-pr266\") pod \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") "
Jul 20 16:20:39.411075 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.411066    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b73bac-1643-4be9-85c3-68ccb2bd60c9-maas-api-tls\") pod \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") "
Jul 20 16:20:39.411283 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.411121    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b73bac-1643-4be9-85c3-68ccb2bd60c9-openshift-service-ca\") pod \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\" (UID: \"07b73bac-1643-4be9-85c3-68ccb2bd60c9\") "
Jul 20 16:20:39.411529 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.411500    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07b73bac-1643-4be9-85c3-68ccb2bd60c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "07b73bac-1643-4be9-85c3-68ccb2bd60c9" (UID: "07b73bac-1643-4be9-85c3-68ccb2bd60c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:20:39.413150 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.413121    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b73bac-1643-4be9-85c3-68ccb2bd60c9-kube-api-access-pr266" (OuterVolumeSpecName: "kube-api-access-pr266") pod "07b73bac-1643-4be9-85c3-68ccb2bd60c9" (UID: "07b73bac-1643-4be9-85c3-68ccb2bd60c9"). InnerVolumeSpecName "kube-api-access-pr266". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:20:39.413150 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.413131    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07b73bac-1643-4be9-85c3-68ccb2bd60c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "07b73bac-1643-4be9-85c3-68ccb2bd60c9" (UID: "07b73bac-1643-4be9-85c3-68ccb2bd60c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:20:39.488355 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.488318    2564 generic.go:356] "Generic (PLEG): container finished" podID="07b73bac-1643-4be9-85c3-68ccb2bd60c9" containerID="65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e" exitCode=0
Jul 20 16:20:39.488531 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.488370    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"
Jul 20 16:20:39.488531 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.488408    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f" event={"ID":"07b73bac-1643-4be9-85c3-68ccb2bd60c9","Type":"ContainerDied","Data":"65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e"}
Jul 20 16:20:39.488531 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.488466    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f" event={"ID":"07b73bac-1643-4be9-85c3-68ccb2bd60c9","Type":"ContainerDied","Data":"f53d22df5771f39a9363e1dc5b272e3582673ac841c4b77ff924dd075f993e77"}
Jul 20 16:20:39.488531 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.488495    2564 scope.go:117] "RemoveContainer" containerID="65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e"
Jul 20 16:20:39.497992 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.497972    2564 scope.go:117] "RemoveContainer" containerID="65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e"
Jul 20 16:20:39.498304 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:20:39.498283    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e\": container with ID starting with 65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e not found: ID does not exist" containerID="65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e"
Jul 20 16:20:39.498355 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.498314    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e"} err="failed to get container status \"65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e\": rpc error: code = NotFound desc = could not find container \"65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e\": container with ID starting with 65f2e9062e5fa9a18908bd1959e46951614087cca8f07cd2ae858dc799cac38e not found: ID does not exist"
Jul 20 16:20:39.510444 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.510407    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"]
Jul 20 16:20:39.512314 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.512244    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr266\" (UniqueName: \"kubernetes.io/projected/07b73bac-1643-4be9-85c3-68ccb2bd60c9-kube-api-access-pr266\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:20:39.512314 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.512277    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b73bac-1643-4be9-85c3-68ccb2bd60c9-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:20:39.512314 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.512291    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b73bac-1643-4be9-85c3-68ccb2bd60c9-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:20:39.514209 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.514183    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6679b04f-86dcc764fc-nmq4f"]
Jul 20 16:20:39.868471 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:39.868434    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07b73bac-1643-4be9-85c3-68ccb2bd60c9" path="/var/lib/kubelet/pods/07b73bac-1643-4be9-85c3-68ccb2bd60c9/volumes"
Jul 20 16:20:43.461868 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:43.461829    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"]
Jul 20 16:20:43.463965 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:43.463940    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6679b04f-8tjdk"]
Jul 20 16:20:43.464743 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:43.464713    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:43.637834 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:43.637777    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:43.854081 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:43.854038    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:43.868638 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:43.868608    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e8e6597-26d6-4c10-a60b-6ef857f9478f" path="/var/lib/kubelet/pods/9e8e6597-26d6-4c10-a60b-6ef857f9478f/volumes"
Jul 20 16:20:44.322788 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:44.322748    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:44.420677 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:44.420643    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:44.518218 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:44.518134    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:45.921356 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:45.921314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:51.780007 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:51.779973    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:51.885377 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:51.885341    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:51.982164 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:51.982124    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:52.080045 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:52.079958    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:20:52.474651 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:20:52.474536    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:18.450639 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:18.450590    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:24.981275 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:24.981242    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:25.087231 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:25.087196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:25.185862 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:25.185829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:41.115647 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:41.115601    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:47.055653 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:47.055593    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:47.113710 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:47.113651    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:47.188269 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:47.188220    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:47.282750 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:47.282714    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:57.856360 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:57.856273    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:21:58.734293 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:21:58.734255    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:05.761837 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:05.761805    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:05.859070 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:05.859035    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:05.959838 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:05.959800    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:06.059505 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:06.059471    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:06.159719 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:06.159662    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:06.963623 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:06.963587    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:07.056925 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:07.056875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:07.158912 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:07.158873    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:09.004334 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:09.004293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:14.800635 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:14.800598    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:14.900454 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:14.900417    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:15.647779 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:15.647743    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:21.517894 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:21.517754    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:21.618149 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:21.618113    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:21.782675 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:21.782641    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:22.133480 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:22.133313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:22.223742 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:22.223713    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:24.642429 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:24.642396    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:25.085582 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.085544    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:25.177437 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.177405    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:25.308382 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.308346    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"]
Jul 20 16:22:25.308837 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.308813    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b73bac-1643-4be9-85c3-68ccb2bd60c9" containerName="maas-api"
Jul 20 16:22:25.308837 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.308836    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b73bac-1643-4be9-85c3-68ccb2bd60c9" containerName="maas-api"
Jul 20 16:22:25.309020 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.308854    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8e6597-26d6-4c10-a60b-6ef857f9478f" containerName="revoke-keys"
Jul 20 16:22:25.309020 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.308863    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8e6597-26d6-4c10-a60b-6ef857f9478f" containerName="revoke-keys"
Jul 20 16:22:25.309020 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.308953    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b73bac-1643-4be9-85c3-68ccb2bd60c9" containerName="maas-api"
Jul 20 16:22:25.309020 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.308964    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8e6597-26d6-4c10-a60b-6ef857f9478f" containerName="revoke-keys"
Jul 20 16:22:25.311769 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.311734    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:25.315526 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.315502    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0df208ce-serving-cert\""
Jul 20 16:22:25.325487 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.325456    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"]
Jul 20 16:22:25.349397 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.349297    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:25.451894 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.451858    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f931858-c45d-441c-82ab-af8f8ff32833-openshift-service-ca\") pod \"maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:25.452083 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.451905    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vcfb\" (UniqueName: \"kubernetes.io/projected/3f931858-c45d-441c-82ab-af8f8ff32833-kube-api-access-7vcfb\") pod \"maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:25.452083 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.451982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f931858-c45d-441c-82ab-af8f8ff32833-maas-api-tls\") pod \"maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:25.552956 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.552913    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f931858-c45d-441c-82ab-af8f8ff32833-openshift-service-ca\") pod \"maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:25.552956 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.552965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vcfb\" (UniqueName: \"kubernetes.io/projected/3f931858-c45d-441c-82ab-af8f8ff32833-kube-api-access-7vcfb\") pod \"maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:25.553227 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.552994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f931858-c45d-441c-82ab-af8f8ff32833-maas-api-tls\") pod \"maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:25.553227 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:22:25.553116    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-serving-cert: secret "maas-api-e2e-mt-0df208ce-serving-cert" not found
Jul 20 16:22:25.553227 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:22:25.553185    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f931858-c45d-441c-82ab-af8f8ff32833-maas-api-tls podName:3f931858-c45d-441c-82ab-af8f8ff32833 nodeName:}" failed. No retries permitted until 2026-07-20 16:22:26.053165997 +0000 UTC m=+1910.708429252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3f931858-c45d-441c-82ab-af8f8ff32833-maas-api-tls") pod "maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z" (UID: "3f931858-c45d-441c-82ab-af8f8ff32833") : secret "maas-api-e2e-mt-0df208ce-serving-cert" not found
Jul 20 16:22:25.553663 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.553637    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f931858-c45d-441c-82ab-af8f8ff32833-openshift-service-ca\") pod \"maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:25.563591 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.563560    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vcfb\" (UniqueName: \"kubernetes.io/projected/3f931858-c45d-441c-82ab-af8f8ff32833-kube-api-access-7vcfb\") pod \"maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:25.719307 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.719210    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:25.957983 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:25.957935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:26.058023 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:26.057984    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f931858-c45d-441c-82ab-af8f8ff32833-maas-api-tls\") pod \"maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:26.060683 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:26.060646    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f931858-c45d-441c-82ab-af8f8ff32833-maas-api-tls\") pod \"maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:26.115956 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:26.115924    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:26.221935 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:26.221893    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:26.365760 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:26.365732    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"]
Jul 20 16:22:26.368318 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:22:26.368283    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f931858_c45d_441c_82ab_af8f8ff32833.slice/crio-715ab0b2a0c14aa53c36a5517dc6d7822fa8bfcf2f66f8a506d14cdc204af3fa WatchSource:0}: Error finding container 715ab0b2a0c14aa53c36a5517dc6d7822fa8bfcf2f66f8a506d14cdc204af3fa: Status 404 returned error can't find the container with id 715ab0b2a0c14aa53c36a5517dc6d7822fa8bfcf2f66f8a506d14cdc204af3fa
Jul 20 16:22:26.979475 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:26.979442    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z" event={"ID":"3f931858-c45d-441c-82ab-af8f8ff32833","Type":"ContainerStarted","Data":"715ab0b2a0c14aa53c36a5517dc6d7822fa8bfcf2f66f8a506d14cdc204af3fa"}
Jul 20 16:22:27.985287 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:27.985250    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z" event={"ID":"3f931858-c45d-441c-82ab-af8f8ff32833","Type":"ContainerStarted","Data":"d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032"}
Jul 20 16:22:27.985763 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:27.985378    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:28.021340 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:28.021286    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z" podStartSLOduration=2.472413098 podStartE2EDuration="3.02126755s" totalImagesPullingTime="548.854452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:22:28.020523033 +0000 UTC m=+1912.675786308" watchObservedRunningTime="2026-07-20 16:22:28.02126755 +0000 UTC m=+1912.676530825"
Jul 20 16:22:29.546800 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:29.546764    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:30.231402 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:30.231325    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:30.348875 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:30.348844    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:30.673484 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:30.673453    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:32.445390 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:32.445293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:33.995144 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:33.995117    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:36.533120 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:36.533085    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:36.634370 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:36.634336    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:36.898416 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:36.898332    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"]
Jul 20 16:22:36.901859 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:36.901840    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"
Jul 20 16:22:36.907056 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:36.907025    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:22:36.913220 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:36.913189    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"]
Jul 20 16:22:36.952329 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:36.952295    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72038d00-eeb1-44ad-bb7e-3eaf756a9150-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7\" (UID: \"72038d00-eeb1-44ad-bb7e-3eaf756a9150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"
Jul 20 16:22:37.053524 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:37.053482    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72038d00-eeb1-44ad-bb7e-3eaf756a9150-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7\" (UID: \"72038d00-eeb1-44ad-bb7e-3eaf756a9150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"
Jul 20 16:22:37.054227 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:37.054208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72038d00-eeb1-44ad-bb7e-3eaf756a9150-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7\" (UID: \"72038d00-eeb1-44ad-bb7e-3eaf756a9150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"
Jul 20 16:22:37.213610 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:37.213509    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"
Jul 20 16:22:37.350563 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:37.350529    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"]
Jul 20 16:22:37.353393 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:22:37.353360    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72038d00_eeb1_44ad_bb7e_3eaf756a9150.slice/crio-c5081219d04abb6c409bcff05a945767acac3eecaa44469b4426ab8aa192607c WatchSource:0}: Error finding container c5081219d04abb6c409bcff05a945767acac3eecaa44469b4426ab8aa192607c: Status 404 returned error can't find the container with id c5081219d04abb6c409bcff05a945767acac3eecaa44469b4426ab8aa192607c
Jul 20 16:22:38.034954 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:38.034923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7" event={"ID":"72038d00-eeb1-44ad-bb7e-3eaf756a9150","Type":"ContainerStarted","Data":"0fc0163403324d7fee17b7c90adfb0fc160a0b8e6e2ae02b43ec05ab1f3a040e"}
Jul 20 16:22:38.035277 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:38.034976    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7" event={"ID":"72038d00-eeb1-44ad-bb7e-3eaf756a9150","Type":"ContainerStarted","Data":"c5081219d04abb6c409bcff05a945767acac3eecaa44469b4426ab8aa192607c"}
Jul 20 16:22:39.041022 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:39.040983    2564 generic.go:356] "Generic (PLEG): container finished" podID="72038d00-eeb1-44ad-bb7e-3eaf756a9150" containerID="0fc0163403324d7fee17b7c90adfb0fc160a0b8e6e2ae02b43ec05ab1f3a040e" exitCode=0
Jul 20 16:22:39.041439 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:39.041072    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7" event={"ID":"72038d00-eeb1-44ad-bb7e-3eaf756a9150","Type":"ContainerDied","Data":"0fc0163403324d7fee17b7c90adfb0fc160a0b8e6e2ae02b43ec05ab1f3a040e"}
Jul 20 16:22:39.172772 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:39.172740    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"
Jul 20 16:22:39.273965 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:39.273917    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72038d00-eeb1-44ad-bb7e-3eaf756a9150-maas-api-service-ca\") pod \"72038d00-eeb1-44ad-bb7e-3eaf756a9150\" (UID: \"72038d00-eeb1-44ad-bb7e-3eaf756a9150\") "
Jul 20 16:22:39.274299 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:39.274275    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72038d00-eeb1-44ad-bb7e-3eaf756a9150-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72038d00-eeb1-44ad-bb7e-3eaf756a9150" (UID: "72038d00-eeb1-44ad-bb7e-3eaf756a9150"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:22:39.375453 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:39.375365    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72038d00-eeb1-44ad-bb7e-3eaf756a9150-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:22:40.047991 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:40.047958    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"
Jul 20 16:22:40.048517 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:40.047965    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7" event={"ID":"72038d00-eeb1-44ad-bb7e-3eaf756a9150","Type":"ContainerDied","Data":"c5081219d04abb6c409bcff05a945767acac3eecaa44469b4426ab8aa192607c"}
Jul 20 16:22:40.048517 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:40.048071    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5081219d04abb6c409bcff05a945767acac3eecaa44469b4426ab8aa192607c"
Jul 20 16:22:47.416127 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.416089    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"]
Jul 20 16:22:47.416557 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.416333    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z" podUID="3f931858-c45d-441c-82ab-af8f8ff32833" containerName="maas-api" containerID="cri-o://d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032" gracePeriod=30
Jul 20 16:22:47.447047 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.447015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:47.668722 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.668637    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:47.748508 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.748476    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f931858-c45d-441c-82ab-af8f8ff32833-openshift-service-ca\") pod \"3f931858-c45d-441c-82ab-af8f8ff32833\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") "
Jul 20 16:22:47.748749 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.748546    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vcfb\" (UniqueName: \"kubernetes.io/projected/3f931858-c45d-441c-82ab-af8f8ff32833-kube-api-access-7vcfb\") pod \"3f931858-c45d-441c-82ab-af8f8ff32833\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") "
Jul 20 16:22:47.748749 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.748593    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f931858-c45d-441c-82ab-af8f8ff32833-maas-api-tls\") pod \"3f931858-c45d-441c-82ab-af8f8ff32833\" (UID: \"3f931858-c45d-441c-82ab-af8f8ff32833\") "
Jul 20 16:22:47.748957 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.748930    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f931858-c45d-441c-82ab-af8f8ff32833-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3f931858-c45d-441c-82ab-af8f8ff32833" (UID: "3f931858-c45d-441c-82ab-af8f8ff32833"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:22:47.750726 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.750705    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f931858-c45d-441c-82ab-af8f8ff32833-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3f931858-c45d-441c-82ab-af8f8ff32833" (UID: "3f931858-c45d-441c-82ab-af8f8ff32833"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:22:47.750857 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.750834    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f931858-c45d-441c-82ab-af8f8ff32833-kube-api-access-7vcfb" (OuterVolumeSpecName: "kube-api-access-7vcfb") pod "3f931858-c45d-441c-82ab-af8f8ff32833" (UID: "3f931858-c45d-441c-82ab-af8f8ff32833"). InnerVolumeSpecName "kube-api-access-7vcfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:22:47.850072 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.850032    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f931858-c45d-441c-82ab-af8f8ff32833-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:22:47.850072 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.850067    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f931858-c45d-441c-82ab-af8f8ff32833-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:22:47.850072 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:47.850078    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vcfb\" (UniqueName: \"kubernetes.io/projected/3f931858-c45d-441c-82ab-af8f8ff32833-kube-api-access-7vcfb\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:22:48.084062 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:48.084025    2564 generic.go:356] "Generic (PLEG): container finished" podID="3f931858-c45d-441c-82ab-af8f8ff32833" containerID="d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032" exitCode=0
Jul 20 16:22:48.084062 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:48.084065    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"
Jul 20 16:22:48.084280 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:48.084079    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z" event={"ID":"3f931858-c45d-441c-82ab-af8f8ff32833","Type":"ContainerDied","Data":"d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032"}
Jul 20 16:22:48.084280 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:48.084129    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z" event={"ID":"3f931858-c45d-441c-82ab-af8f8ff32833","Type":"ContainerDied","Data":"715ab0b2a0c14aa53c36a5517dc6d7822fa8bfcf2f66f8a506d14cdc204af3fa"}
Jul 20 16:22:48.084280 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:48.084153    2564 scope.go:117] "RemoveContainer" containerID="d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032"
Jul 20 16:22:48.093079 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:48.093058    2564 scope.go:117] "RemoveContainer" containerID="d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032"
Jul 20 16:22:48.093369 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:22:48.093351    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032\": container with ID starting with d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032 not found: ID does not exist" containerID="d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032"
Jul 20 16:22:48.093414 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:48.093378    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032"} err="failed to get container status \"d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032\": rpc error: code = NotFound desc = could not find container \"d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032\": container with ID starting with d1c0ef9e213b78870c7b58388f9a2fdc5438d884a23a022bfc4e039b745d9032 not found: ID does not exist"
Jul 20 16:22:48.101346 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:48.101319    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"]
Jul 20 16:22:48.105049 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:48.105025    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0df208ce-8fd86fc95-mlh7z"]
Jul 20 16:22:49.869083 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:49.869051    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f931858-c45d-441c-82ab-af8f8ff32833" path="/var/lib/kubelet/pods/3f931858-c45d-441c-82ab-af8f8ff32833/volumes"
Jul 20 16:22:52.012828 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:52.012789    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:52.025576 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:52.025541    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"]
Jul 20 16:22:52.028458 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:52.028431    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0df208ce-2q8h7"]
Jul 20 16:22:52.400047 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:52.399994    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:53.869777 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:53.869739    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72038d00-eeb1-44ad-bb7e-3eaf756a9150" path="/var/lib/kubelet/pods/72038d00-eeb1-44ad-bb7e-3eaf756a9150/volumes"
Jul 20 16:22:58.249932 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:58.249895    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:58.350217 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:58.350185    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:58.525399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:58.525368    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:58.746135 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:58.746099    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:58.864069 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:58.863989    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:58.963137 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:58.963106    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:22:59.074300 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:22:59.074264    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:04.795949 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:04.795898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:04.904815 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:04.904782    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:04.995346 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:04.995314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:05.161459 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:05.161361    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:11.102175 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:11.102140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:11.162678 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:11.162644    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:18.008048 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:18.008005    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:18.321083 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:18.321028    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:18.845828 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:18.845794    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:21.258213 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.258174    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:21.361869 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.361837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:21.491796 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.491758    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"]
Jul 20 16:23:21.492283 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.492264    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72038d00-eeb1-44ad-bb7e-3eaf756a9150" containerName="revoke-keys"
Jul 20 16:23:21.492348 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.492289    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="72038d00-eeb1-44ad-bb7e-3eaf756a9150" containerName="revoke-keys"
Jul 20 16:23:21.492348 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.492334    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f931858-c45d-441c-82ab-af8f8ff32833" containerName="maas-api"
Jul 20 16:23:21.492348 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.492344    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f931858-c45d-441c-82ab-af8f8ff32833" containerName="maas-api"
Jul 20 16:23:21.492494 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.492425    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="72038d00-eeb1-44ad-bb7e-3eaf756a9150" containerName="revoke-keys"
Jul 20 16:23:21.492494 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.492437    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f931858-c45d-441c-82ab-af8f8ff32833" containerName="maas-api"
Jul 20 16:23:21.496167 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.496143    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:21.498386 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.498360    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-aef3b1b4-serving-cert\""
Jul 20 16:23:21.505333 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.505309    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"]
Jul 20 16:23:21.521070 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.521021    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:21.540868 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.540835    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de47b9d-683d-44da-ad09-49f2e29a1d85-maas-api-tls\") pod \"maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:21.541063 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.540877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzlrg\" (UniqueName: \"kubernetes.io/projected/4de47b9d-683d-44da-ad09-49f2e29a1d85-kube-api-access-tzlrg\") pod \"maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:21.541063 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.540907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de47b9d-683d-44da-ad09-49f2e29a1d85-openshift-service-ca\") pod \"maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:21.642198 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.642158    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de47b9d-683d-44da-ad09-49f2e29a1d85-maas-api-tls\") pod \"maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:21.642394 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.642216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzlrg\" (UniqueName: \"kubernetes.io/projected/4de47b9d-683d-44da-ad09-49f2e29a1d85-kube-api-access-tzlrg\") pod \"maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:21.642394 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.642269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de47b9d-683d-44da-ad09-49f2e29a1d85-openshift-service-ca\") pod \"maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:21.642394 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:23:21.642326    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-serving-cert: secret "maas-api-e2e-mt-aef3b1b4-serving-cert" not found
Jul 20 16:23:21.642504 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:23:21.642417    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de47b9d-683d-44da-ad09-49f2e29a1d85-maas-api-tls podName:4de47b9d-683d-44da-ad09-49f2e29a1d85 nodeName:}" failed. No retries permitted until 2026-07-20 16:23:22.142395946 +0000 UTC m=+1966.797659215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4de47b9d-683d-44da-ad09-49f2e29a1d85-maas-api-tls") pod "maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v" (UID: "4de47b9d-683d-44da-ad09-49f2e29a1d85") : secret "maas-api-e2e-mt-aef3b1b4-serving-cert" not found
Jul 20 16:23:21.642959 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.642939    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de47b9d-683d-44da-ad09-49f2e29a1d85-openshift-service-ca\") pod \"maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:21.651918 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.651891    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzlrg\" (UniqueName: \"kubernetes.io/projected/4de47b9d-683d-44da-ad09-49f2e29a1d85-kube-api-access-tzlrg\") pod \"maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:21.996900 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:21.996790    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:22.100067 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:22.100024    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:22.147136 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:22.147103    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de47b9d-683d-44da-ad09-49f2e29a1d85-maas-api-tls\") pod \"maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:22.149536 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:22.149504    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de47b9d-683d-44da-ad09-49f2e29a1d85-maas-api-tls\") pod \"maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:22.153341 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:22.153310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:22.409631 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:22.409593    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:22.549235 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:22.549199    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"]
Jul 20 16:23:22.552651 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:23:22.552618    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de47b9d_683d_44da_ad09_49f2e29a1d85.slice/crio-36caf6025b110957347907200da563e1c49dd271d9c83b7e252603222ec66dec WatchSource:0}: Error finding container 36caf6025b110957347907200da563e1c49dd271d9c83b7e252603222ec66dec: Status 404 returned error can't find the container with id 36caf6025b110957347907200da563e1c49dd271d9c83b7e252603222ec66dec
Jul 20 16:23:23.249606 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:23.249572    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v" event={"ID":"4de47b9d-683d-44da-ad09-49f2e29a1d85","Type":"ContainerStarted","Data":"bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730"}
Jul 20 16:23:23.249807 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:23.249628    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v" event={"ID":"4de47b9d-683d-44da-ad09-49f2e29a1d85","Type":"ContainerStarted","Data":"36caf6025b110957347907200da563e1c49dd271d9c83b7e252603222ec66dec"}
Jul 20 16:23:23.249807 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:23.249734    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:23.266398 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:23.266342    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v" podStartSLOduration=1.682711196 podStartE2EDuration="2.266321214s" totalImagesPullingTime="583.610018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:23:23.265366962 +0000 UTC m=+1967.920630239" watchObservedRunningTime="2026-07-20 16:23:23.266321214 +0000 UTC m=+1967.921584492"
Jul 20 16:23:25.758353 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:25.758323    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:23:29.259914 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:29.259834    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:23:33.793924 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:23:33.793889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:24:04.228143 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:04.228106    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"]
Jul 20 16:24:04.231478 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:04.231461    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"
Jul 20 16:24:04.234159 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:04.234138    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:24:04.240645 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:04.240614    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"]
Jul 20 16:24:04.316572 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:04.316527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc49259-ba32-4470-b4de-02bc88cdfd4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76\" (UID: \"4bc49259-ba32-4470-b4de-02bc88cdfd4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"
Jul 20 16:24:04.417150 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:04.417115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc49259-ba32-4470-b4de-02bc88cdfd4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76\" (UID: \"4bc49259-ba32-4470-b4de-02bc88cdfd4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"
Jul 20 16:24:04.417810 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:04.417788    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc49259-ba32-4470-b4de-02bc88cdfd4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76\" (UID: \"4bc49259-ba32-4470-b4de-02bc88cdfd4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"
Jul 20 16:24:04.542268 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:04.542236    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"
Jul 20 16:24:04.634271 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:04.634227    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:24:04.701263 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:04.701242    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"]
Jul 20 16:24:04.703864 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:24:04.703832    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc49259_ba32_4470_b4de_02bc88cdfd4e.slice/crio-edae1102d1bf151ba8b11d0236d024d48686ea712ba8a2c0040bfbedd7ce8f5d WatchSource:0}: Error finding container edae1102d1bf151ba8b11d0236d024d48686ea712ba8a2c0040bfbedd7ce8f5d: Status 404 returned error can't find the container with id edae1102d1bf151ba8b11d0236d024d48686ea712ba8a2c0040bfbedd7ce8f5d
Jul 20 16:24:05.437316 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:05.437219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76" event={"ID":"4bc49259-ba32-4470-b4de-02bc88cdfd4e","Type":"ContainerStarted","Data":"16650c1180f3de37077ff4a2240a00b1ebdafabaae90f186f4c9dfdf25e06be7"}
Jul 20 16:24:05.437316 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:05.437294    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76" event={"ID":"4bc49259-ba32-4470-b4de-02bc88cdfd4e","Type":"ContainerStarted","Data":"edae1102d1bf151ba8b11d0236d024d48686ea712ba8a2c0040bfbedd7ce8f5d"}
Jul 20 16:24:05.453249 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:05.453196    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76" podStartSLOduration=1.453179124 podStartE2EDuration="1.453179124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:24:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:24:05.452140058 +0000 UTC m=+2010.107403333" watchObservedRunningTime="2026-07-20 16:24:05.453179124 +0000 UTC m=+2010.108442400"
Jul 20 16:24:06.443533 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:06.443498    2564 generic.go:356] "Generic (PLEG): container finished" podID="4bc49259-ba32-4470-b4de-02bc88cdfd4e" containerID="16650c1180f3de37077ff4a2240a00b1ebdafabaae90f186f4c9dfdf25e06be7" exitCode=0
Jul 20 16:24:06.443948 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:06.443545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76" event={"ID":"4bc49259-ba32-4470-b4de-02bc88cdfd4e","Type":"ContainerDied","Data":"16650c1180f3de37077ff4a2240a00b1ebdafabaae90f186f4c9dfdf25e06be7"}
Jul 20 16:24:07.584144 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:07.584117    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"
Jul 20 16:24:07.640220 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:07.640184    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc49259-ba32-4470-b4de-02bc88cdfd4e-maas-api-service-ca\") pod \"4bc49259-ba32-4470-b4de-02bc88cdfd4e\" (UID: \"4bc49259-ba32-4470-b4de-02bc88cdfd4e\") "
Jul 20 16:24:07.640542 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:07.640517    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bc49259-ba32-4470-b4de-02bc88cdfd4e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4bc49259-ba32-4470-b4de-02bc88cdfd4e" (UID: "4bc49259-ba32-4470-b4de-02bc88cdfd4e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:24:07.740905 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:07.740813    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc49259-ba32-4470-b4de-02bc88cdfd4e-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:24:08.454278 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:08.454243    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76" event={"ID":"4bc49259-ba32-4470-b4de-02bc88cdfd4e","Type":"ContainerDied","Data":"edae1102d1bf151ba8b11d0236d024d48686ea712ba8a2c0040bfbedd7ce8f5d"}
Jul 20 16:24:08.454457 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:08.454294    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edae1102d1bf151ba8b11d0236d024d48686ea712ba8a2c0040bfbedd7ce8f5d"
Jul 20 16:24:08.454457 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:08.454260    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"
Jul 20 16:24:19.880209 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:19.880174    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"]
Jul 20 16:24:19.880706 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:19.880376    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v" podUID="4de47b9d-683d-44da-ad09-49f2e29a1d85" containerName="maas-api" containerID="cri-o://bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730" gracePeriod=30
Jul 20 16:24:19.901285 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:19.901257    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:24:20.123680 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.123654    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:24:20.239466 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.239375    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de47b9d-683d-44da-ad09-49f2e29a1d85-openshift-service-ca\") pod \"4de47b9d-683d-44da-ad09-49f2e29a1d85\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") "
Jul 20 16:24:20.239466 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.239424    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de47b9d-683d-44da-ad09-49f2e29a1d85-maas-api-tls\") pod \"4de47b9d-683d-44da-ad09-49f2e29a1d85\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") "
Jul 20 16:24:20.239466 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.239461    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzlrg\" (UniqueName: \"kubernetes.io/projected/4de47b9d-683d-44da-ad09-49f2e29a1d85-kube-api-access-tzlrg\") pod \"4de47b9d-683d-44da-ad09-49f2e29a1d85\" (UID: \"4de47b9d-683d-44da-ad09-49f2e29a1d85\") "
Jul 20 16:24:20.239800 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.239765    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de47b9d-683d-44da-ad09-49f2e29a1d85-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4de47b9d-683d-44da-ad09-49f2e29a1d85" (UID: "4de47b9d-683d-44da-ad09-49f2e29a1d85"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:24:20.241554 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.241527    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de47b9d-683d-44da-ad09-49f2e29a1d85-kube-api-access-tzlrg" (OuterVolumeSpecName: "kube-api-access-tzlrg") pod "4de47b9d-683d-44da-ad09-49f2e29a1d85" (UID: "4de47b9d-683d-44da-ad09-49f2e29a1d85"). InnerVolumeSpecName "kube-api-access-tzlrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:24:20.241641 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.241548    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4de47b9d-683d-44da-ad09-49f2e29a1d85-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4de47b9d-683d-44da-ad09-49f2e29a1d85" (UID: "4de47b9d-683d-44da-ad09-49f2e29a1d85"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:24:20.340172 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.340119    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de47b9d-683d-44da-ad09-49f2e29a1d85-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:24:20.340172 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.340164    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzlrg\" (UniqueName: \"kubernetes.io/projected/4de47b9d-683d-44da-ad09-49f2e29a1d85-kube-api-access-tzlrg\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:24:20.340172 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.340180    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de47b9d-683d-44da-ad09-49f2e29a1d85-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:24:20.512602 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.512515    2564 generic.go:356] "Generic (PLEG): container finished" podID="4de47b9d-683d-44da-ad09-49f2e29a1d85" containerID="bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730" exitCode=0
Jul 20 16:24:20.512602 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.512559    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"
Jul 20 16:24:20.512602 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.512579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v" event={"ID":"4de47b9d-683d-44da-ad09-49f2e29a1d85","Type":"ContainerDied","Data":"bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730"}
Jul 20 16:24:20.512891 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.512622    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v" event={"ID":"4de47b9d-683d-44da-ad09-49f2e29a1d85","Type":"ContainerDied","Data":"36caf6025b110957347907200da563e1c49dd271d9c83b7e252603222ec66dec"}
Jul 20 16:24:20.512891 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.512644    2564 scope.go:117] "RemoveContainer" containerID="bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730"
Jul 20 16:24:20.522584 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.522561    2564 scope.go:117] "RemoveContainer" containerID="bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730"
Jul 20 16:24:20.522885 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:24:20.522856    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730\": container with ID starting with bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730 not found: ID does not exist" containerID="bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730"
Jul 20 16:24:20.522999 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.522904    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730"} err="failed to get container status \"bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730\": rpc error: code = NotFound desc = could not find container \"bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730\": container with ID starting with bde2581920bf099b1a3c076c37069c1ce69513945778c60748b8f486f5e72730 not found: ID does not exist"
Jul 20 16:24:20.539354 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.539324    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"]
Jul 20 16:24:20.542549 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:20.542524    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aef3b1b4-598c59b44f-dmr9v"]
Jul 20 16:24:21.868458 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:21.868419    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4de47b9d-683d-44da-ad09-49f2e29a1d85" path="/var/lib/kubelet/pods/4de47b9d-683d-44da-ad09-49f2e29a1d85/volumes"
Jul 20 16:24:24.320393 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:24.320358    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:24:24.336702 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:24.336643    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"]
Jul 20 16:24:24.339759 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:24.339736    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aef3b1b4-k9c76"]
Jul 20 16:24:25.870142 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:25.870106    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bc49259-ba32-4470-b4de-02bc88cdfd4e" path="/var/lib/kubelet/pods/4bc49259-ba32-4470-b4de-02bc88cdfd4e/volumes"
Jul 20 16:24:26.584403 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:26.584366    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:24:31.666740 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:31.666682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:24:35.999140 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:35.999101    2564 scope.go:117] "RemoveContainer" containerID="2f329cde3cc255a9aaf2c1f7785e6b4ed4549a9e6c18921f2453fa59b90731e4"
Jul 20 16:24:48.053281 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:48.053224    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:24:48.152212 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:24:48.152182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:08.643172 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:08.643074    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:09.084619 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:09.084582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:14.870036 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:14.869994    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:14.927280 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:14.927244    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:15.198569 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:15.198481    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:15.457174 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:15.457076    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:16.614663 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:16.614623    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:16.708425 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:16.708389    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:16.808392 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:16.808355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:17.636990 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:17.636950    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:17.737747 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:17.737712    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:17.843034 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:17.843003    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:19.368038 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:19.368009    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:21.816459 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:21.816427    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:25.997639 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:25.997606    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:26.097286 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:26.097245    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:26.743271 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:26.743237    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:33.070250 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:33.070217    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:33.124331 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:33.124285    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:33.217308 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:33.217264    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:33.485811 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:33.485683    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:35.913969 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:35.913941    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:25:35.915887 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:35.915862    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:25:36.020978 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:36.020949    2564 scope.go:117] "RemoveContainer" containerID="0e39d6115e984e16b6fb5fab7e92f4378464d8a78d3cae992707a6d3122a3c74"
Jul 20 16:25:43.816475 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:43.816441    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:25:44.176222 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:25:44.176120    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:05.182435 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:05.182402    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:11.537050 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:11.537010    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:11.988181 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:11.988087    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:12.043868 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:12.043829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:12.358563 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:12.358526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:12.671404 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:12.671317    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:13.019574 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:13.019537    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:15.188796 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.188755    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:15.280721 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.280649    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:15.394660 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.394615    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"]
Jul 20 16:26:15.394992 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.394978    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4de47b9d-683d-44da-ad09-49f2e29a1d85" containerName="maas-api"
Jul 20 16:26:15.394992 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.394993    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4de47b9d-683d-44da-ad09-49f2e29a1d85" containerName="maas-api"
Jul 20 16:26:15.395077 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.395006    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc49259-ba32-4470-b4de-02bc88cdfd4e" containerName="revoke-keys"
Jul 20 16:26:15.395077 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.395012    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc49259-ba32-4470-b4de-02bc88cdfd4e" containerName="revoke-keys"
Jul 20 16:26:15.395077 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.395065    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4de47b9d-683d-44da-ad09-49f2e29a1d85" containerName="maas-api"
Jul 20 16:26:15.395077 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.395073    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bc49259-ba32-4470-b4de-02bc88cdfd4e" containerName="revoke-keys"
Jul 20 16:26:15.397917 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.397898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.399841 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.399818    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-f3d110-serving-cert\""
Jul 20 16:26:15.410664 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.410635    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"]
Jul 20 16:26:15.436122 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.436075    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:15.526729 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.526670    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpthd\" (UniqueName: \"kubernetes.io/projected/127dea28-7035-4a63-b72d-618bfa9bad46-kube-api-access-lpthd\") pod \"maas-api-e2e-inf-a-f3d110-55868445bd-fblj8\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.526900 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.526782    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/127dea28-7035-4a63-b72d-618bfa9bad46-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f3d110-55868445bd-fblj8\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.526900 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.526809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/127dea28-7035-4a63-b72d-618bfa9bad46-maas-api-tls\") pod \"maas-api-e2e-inf-a-f3d110-55868445bd-fblj8\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.556953 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.556915    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:15.628071 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.628031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/127dea28-7035-4a63-b72d-618bfa9bad46-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f3d110-55868445bd-fblj8\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.628272 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.628081    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/127dea28-7035-4a63-b72d-618bfa9bad46-maas-api-tls\") pod \"maas-api-e2e-inf-a-f3d110-55868445bd-fblj8\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.628272 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.628137    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpthd\" (UniqueName: \"kubernetes.io/projected/127dea28-7035-4a63-b72d-618bfa9bad46-kube-api-access-lpthd\") pod \"maas-api-e2e-inf-a-f3d110-55868445bd-fblj8\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.628669 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.628648    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/127dea28-7035-4a63-b72d-618bfa9bad46-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f3d110-55868445bd-fblj8\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.630570 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.630541    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/127dea28-7035-4a63-b72d-618bfa9bad46-maas-api-tls\") pod \"maas-api-e2e-inf-a-f3d110-55868445bd-fblj8\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.640408 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.640386    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpthd\" (UniqueName: \"kubernetes.io/projected/127dea28-7035-4a63-b72d-618bfa9bad46-kube-api-access-lpthd\") pod \"maas-api-e2e-inf-a-f3d110-55868445bd-fblj8\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.708668 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.708625    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:15.850425 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.850397    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"]
Jul 20 16:26:15.852852 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:26:15.852820    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod127dea28_7035_4a63_b72d_618bfa9bad46.slice/crio-57030e9c1971ef15bc20fcdfe8b3b3716f0712f9db964b181613ef07b397bb7d WatchSource:0}: Error finding container 57030e9c1971ef15bc20fcdfe8b3b3716f0712f9db964b181613ef07b397bb7d: Status 404 returned error can't find the container with id 57030e9c1971ef15bc20fcdfe8b3b3716f0712f9db964b181613ef07b397bb7d
Jul 20 16:26:15.854147 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.854126    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:26:15.964896 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:15.964864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:16.021071 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:16.021031    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8" event={"ID":"127dea28-7035-4a63-b72d-618bfa9bad46","Type":"ContainerStarted","Data":"57030e9c1971ef15bc20fcdfe8b3b3716f0712f9db964b181613ef07b397bb7d"}
Jul 20 16:26:16.065883 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:16.065791    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:17.027340 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:17.027304    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8" event={"ID":"127dea28-7035-4a63-b72d-618bfa9bad46","Type":"ContainerStarted","Data":"7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2"}
Jul 20 16:26:17.027775 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:17.027399    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:17.053695 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:17.053629    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8" podStartSLOduration=1.4972407030000001 podStartE2EDuration="2.053609781s" totalImagesPullingTime="556.369078ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:26:17.052350714 +0000 UTC m=+2141.707613991" watchObservedRunningTime="2026-07-20 16:26:17.053609781 +0000 UTC m=+2141.708873056"
Jul 20 16:26:19.701666 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:19.701633    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:20.023906 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:20.023876    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:20.345198 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:20.345090    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:20.485284 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:20.485244    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:23.037508 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.037477    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:26:23.354564 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.354482    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:23.457215 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.457180    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"]
Jul 20 16:26:23.461050 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.461026    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.467966 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.467944    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-18d8b0-serving-cert\""
Jul 20 16:26:23.471444 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.471419    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:23.484388 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.484359    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"]
Jul 20 16:26:23.547747 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.547721    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:23.596712 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.596653    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b5f65b-1264-4eb1-9f22-01cf3f485485-openshift-service-ca\") pod \"maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.596922 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.596770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlwcg\" (UniqueName: \"kubernetes.io/projected/68b5f65b-1264-4eb1-9f22-01cf3f485485-kube-api-access-tlwcg\") pod \"maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.596922 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.596898    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b5f65b-1264-4eb1-9f22-01cf3f485485-maas-api-tls\") pod \"maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.652900 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.652809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:23.697476 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.697435    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b5f65b-1264-4eb1-9f22-01cf3f485485-openshift-service-ca\") pod \"maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.697728 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.697493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwcg\" (UniqueName: \"kubernetes.io/projected/68b5f65b-1264-4eb1-9f22-01cf3f485485-kube-api-access-tlwcg\") pod \"maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.697728 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.697543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b5f65b-1264-4eb1-9f22-01cf3f485485-maas-api-tls\") pod \"maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.698201 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.698176    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b5f65b-1264-4eb1-9f22-01cf3f485485-openshift-service-ca\") pod \"maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.700072 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.700049    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b5f65b-1264-4eb1-9f22-01cf3f485485-maas-api-tls\") pod \"maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.713348 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.713303    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlwcg\" (UniqueName: \"kubernetes.io/projected/68b5f65b-1264-4eb1-9f22-01cf3f485485-kube-api-access-tlwcg\") pod \"maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.773025 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.772981    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:23.926578 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:23.926552    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"]
Jul 20 16:26:23.929195 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:26:23.929162    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68b5f65b_1264_4eb1_9f22_01cf3f485485.slice/crio-3d541bbd0406ed185583c1b5818b8efb893c9a00aec24bcb7534dad2d87a43f7 WatchSource:0}: Error finding container 3d541bbd0406ed185583c1b5818b8efb893c9a00aec24bcb7534dad2d87a43f7: Status 404 returned error can't find the container with id 3d541bbd0406ed185583c1b5818b8efb893c9a00aec24bcb7534dad2d87a43f7
Jul 20 16:26:24.059994 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:24.059945    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf" event={"ID":"68b5f65b-1264-4eb1-9f22-01cf3f485485","Type":"ContainerStarted","Data":"3d541bbd0406ed185583c1b5818b8efb893c9a00aec24bcb7534dad2d87a43f7"}
Jul 20 16:26:24.200228 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:24.200136    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:24.303082 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:24.303048    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:25.066082 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:25.066044    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf" event={"ID":"68b5f65b-1264-4eb1-9f22-01cf3f485485","Type":"ContainerStarted","Data":"795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35"}
Jul 20 16:26:25.066499 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:25.066150    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:25.083922 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:25.083851    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf" podStartSLOduration=1.534013715 podStartE2EDuration="2.083832621s" totalImagesPullingTime="549.818906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:26:25.083631907 +0000 UTC m=+2149.738895185" watchObservedRunningTime="2026-07-20 16:26:25.083832621 +0000 UTC m=+2149.739095900"
Jul 20 16:26:27.772639 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:27.772610    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:31.076080 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:31.076052    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:26:36.038792 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:36.038761    2564 scope.go:117] "RemoveContainer" containerID="4126da263642a955c48312f21a10e9f96e240323f50f4c04cfab12d1897781e1"
Jul 20 16:26:36.046921 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:36.046900    2564 scope.go:117] "RemoveContainer" containerID="5ac16c1dc6220c71341cab2fc9ce43ae8cd5096c451e4173c1346b89e18be27a"
Jul 20 16:26:36.860808 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:36.860772    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:36.966801 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:36.966761    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:37.693732 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:37.693673    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:45.241532 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:45.241498    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:26:45.334750 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:26:45.334715    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:27:32.872034 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:27:32.871995    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:18.116925 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:18.116882    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:19.153914 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.153882    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:19.244041 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.243996    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"]
Jul 20 16:28:19.247300 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.247275    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"
Jul 20 16:28:19.249670 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.249649    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:28:19.254268 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.254232    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"]
Jul 20 16:28:19.322074 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.322036    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z\" (UID: \"5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"
Jul 20 16:28:19.423569 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.423477    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z\" (UID: \"5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"
Jul 20 16:28:19.424127 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.424106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z\" (UID: \"5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"
Jul 20 16:28:19.560699 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.560663    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"
Jul 20 16:28:19.702507 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.702408    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"]
Jul 20 16:28:19.705565 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:28:19.705536    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d6a69b0_a11a_46d5_b13d_fb8c1c3363d2.slice/crio-c506f55a0977895734cb8d624da7981912815a15cea592c1557c22e55dad746b WatchSource:0}: Error finding container c506f55a0977895734cb8d624da7981912815a15cea592c1557c22e55dad746b: Status 404 returned error can't find the container with id c506f55a0977895734cb8d624da7981912815a15cea592c1557c22e55dad746b
Jul 20 16:28:19.836179 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.836150    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:19.935197 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:19.935171    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:20.586905 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:20.586874    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2" containerID="9dd7affb08b1e554dee17b4ebf9c561e90428b9c645361bfd1ed0694c75e3867" exitCode=0
Jul 20 16:28:20.587367 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:20.586913    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z" event={"ID":"5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2","Type":"ContainerDied","Data":"9dd7affb08b1e554dee17b4ebf9c561e90428b9c645361bfd1ed0694c75e3867"}
Jul 20 16:28:20.587367 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:20.586949    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z" event={"ID":"5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2","Type":"ContainerStarted","Data":"c506f55a0977895734cb8d624da7981912815a15cea592c1557c22e55dad746b"}
Jul 20 16:28:21.736855 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:21.736826    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"
Jul 20 16:28:21.741985 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:21.741955    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2-maas-api-service-ca\") pod \"5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2\" (UID: \"5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2\") "
Jul 20 16:28:21.742308 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:21.742283    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2" (UID: "5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:28:21.843099 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:21.843010    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:28:22.596832 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:22.596797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z" event={"ID":"5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2","Type":"ContainerDied","Data":"c506f55a0977895734cb8d624da7981912815a15cea592c1557c22e55dad746b"}
Jul 20 16:28:22.597012 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:22.596837    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"
Jul 20 16:28:22.597012 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:22.596846    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c506f55a0977895734cb8d624da7981912815a15cea592c1557c22e55dad746b"
Jul 20 16:28:29.741553 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:29.741516    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"]
Jul 20 16:28:29.742345 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:29.742306    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8" podUID="127dea28-7035-4a63-b72d-618bfa9bad46" containerName="maas-api" containerID="cri-o://7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2" gracePeriod=30
Jul 20 16:28:29.765857 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:29.765824    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:29.990447 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:29.990422    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:28:30.115359 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.115325    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/127dea28-7035-4a63-b72d-618bfa9bad46-maas-api-tls\") pod \"127dea28-7035-4a63-b72d-618bfa9bad46\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") "
Jul 20 16:28:30.115555 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.115388    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpthd\" (UniqueName: \"kubernetes.io/projected/127dea28-7035-4a63-b72d-618bfa9bad46-kube-api-access-lpthd\") pod \"127dea28-7035-4a63-b72d-618bfa9bad46\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") "
Jul 20 16:28:30.115555 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.115462    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/127dea28-7035-4a63-b72d-618bfa9bad46-openshift-service-ca\") pod \"127dea28-7035-4a63-b72d-618bfa9bad46\" (UID: \"127dea28-7035-4a63-b72d-618bfa9bad46\") "
Jul 20 16:28:30.115870 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.115845    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/127dea28-7035-4a63-b72d-618bfa9bad46-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "127dea28-7035-4a63-b72d-618bfa9bad46" (UID: "127dea28-7035-4a63-b72d-618bfa9bad46"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:28:30.117596 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.117566    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/127dea28-7035-4a63-b72d-618bfa9bad46-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "127dea28-7035-4a63-b72d-618bfa9bad46" (UID: "127dea28-7035-4a63-b72d-618bfa9bad46"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:28:30.117721 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.117568    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/127dea28-7035-4a63-b72d-618bfa9bad46-kube-api-access-lpthd" (OuterVolumeSpecName: "kube-api-access-lpthd") pod "127dea28-7035-4a63-b72d-618bfa9bad46" (UID: "127dea28-7035-4a63-b72d-618bfa9bad46"). InnerVolumeSpecName "kube-api-access-lpthd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:28:30.216474 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.216432    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/127dea28-7035-4a63-b72d-618bfa9bad46-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:28:30.216474 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.216468    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/127dea28-7035-4a63-b72d-618bfa9bad46-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:28:30.216474 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.216477    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpthd\" (UniqueName: \"kubernetes.io/projected/127dea28-7035-4a63-b72d-618bfa9bad46-kube-api-access-lpthd\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:28:30.635457 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.635419    2564 generic.go:356] "Generic (PLEG): container finished" podID="127dea28-7035-4a63-b72d-618bfa9bad46" containerID="7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2" exitCode=0
Jul 20 16:28:30.635663 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.635466    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"
Jul 20 16:28:30.635663 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.635502    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8" event={"ID":"127dea28-7035-4a63-b72d-618bfa9bad46","Type":"ContainerDied","Data":"7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2"}
Jul 20 16:28:30.635663 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.635566    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8" event={"ID":"127dea28-7035-4a63-b72d-618bfa9bad46","Type":"ContainerDied","Data":"57030e9c1971ef15bc20fcdfe8b3b3716f0712f9db964b181613ef07b397bb7d"}
Jul 20 16:28:30.635663 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.635596    2564 scope.go:117] "RemoveContainer" containerID="7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2"
Jul 20 16:28:30.644943 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.644924    2564 scope.go:117] "RemoveContainer" containerID="7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2"
Jul 20 16:28:30.645203 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:28:30.645181    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2\": container with ID starting with 7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2 not found: ID does not exist" containerID="7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2"
Jul 20 16:28:30.645262 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.645219    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2"} err="failed to get container status \"7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2\": rpc error: code = NotFound desc = could not find container \"7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2\": container with ID starting with 7c3bbf0cd2b1eabd327b9cdc89f23d6f104f96540ba658b3b6b739aff4d9bbd2 not found: ID does not exist"
Jul 20 16:28:30.657773 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.657745    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"]
Jul 20 16:28:30.663202 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:30.663176    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f3d110-55868445bd-fblj8"]
Jul 20 16:28:31.868619 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:31.868583    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="127dea28-7035-4a63-b72d-618bfa9bad46" path="/var/lib/kubelet/pods/127dea28-7035-4a63-b72d-618bfa9bad46/volumes"
Jul 20 16:28:34.331644 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:34.331613    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:34.349115 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:34.349079    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"]
Jul 20 16:28:34.352819 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:34.352793    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f3d110-g9p2z"]
Jul 20 16:28:34.713255 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:34.713165    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:35.869866 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:35.869833    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2" path="/var/lib/kubelet/pods/5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2/volumes"
Jul 20 16:28:40.893889 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:40.893855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:40.959182 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:40.959140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:41.168988 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.168905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:41.409953 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.409921    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:41.573765 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.573720    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"]
Jul 20 16:28:41.574099 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.574083    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127dea28-7035-4a63-b72d-618bfa9bad46" containerName="maas-api"
Jul 20 16:28:41.574099 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.574101    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="127dea28-7035-4a63-b72d-618bfa9bad46" containerName="maas-api"
Jul 20 16:28:41.574211 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.574112    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2" containerName="revoke-keys"
Jul 20 16:28:41.574211 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.574118    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2" containerName="revoke-keys"
Jul 20 16:28:41.574211 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.574177    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="127dea28-7035-4a63-b72d-618bfa9bad46" containerName="maas-api"
Jul 20 16:28:41.574211 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.574185    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d6a69b0-a11a-46d5-b13d-fb8c1c3363d2" containerName="revoke-keys"
Jul 20 16:28:41.577062 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.577044    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"
Jul 20 16:28:41.581827 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.581803    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:28:41.592665 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.592628    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"]
Jul 20 16:28:41.712215 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.712184    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3644dd4f-4a47-4d47-9f3a-9521ce6f66f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw\" (UID: \"3644dd4f-4a47-4d47-9f3a-9521ce6f66f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"
Jul 20 16:28:41.812747 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.812711    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3644dd4f-4a47-4d47-9f3a-9521ce6f66f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw\" (UID: \"3644dd4f-4a47-4d47-9f3a-9521ce6f66f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"
Jul 20 16:28:41.813368 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.813349    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3644dd4f-4a47-4d47-9f3a-9521ce6f66f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw\" (UID: \"3644dd4f-4a47-4d47-9f3a-9521ce6f66f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"
Jul 20 16:28:41.886622 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:41.886528    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"
Jul 20 16:28:42.025111 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:42.025079    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"]
Jul 20 16:28:42.028131 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:28:42.028104    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3644dd4f_4a47_4d47_9f3a_9521ce6f66f4.slice/crio-ec86f3f878693771a7b8771b91bfb6c93a9a8aefec38daab30e475538aeefdfc WatchSource:0}: Error finding container ec86f3f878693771a7b8771b91bfb6c93a9a8aefec38daab30e475538aeefdfc: Status 404 returned error can't find the container with id ec86f3f878693771a7b8771b91bfb6c93a9a8aefec38daab30e475538aeefdfc
Jul 20 16:28:42.297374 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:42.297336    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:42.398906 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:42.398876    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:42.697228 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:42.697177    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw" event={"ID":"3644dd4f-4a47-4d47-9f3a-9521ce6f66f4","Type":"ContainerStarted","Data":"22c76cccb4e14784ae78e9053079e8312170a07128fca2e5c1543a420f2cf4be"}
Jul 20 16:28:42.697384 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:42.697245    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw" event={"ID":"3644dd4f-4a47-4d47-9f3a-9521ce6f66f4","Type":"ContainerStarted","Data":"ec86f3f878693771a7b8771b91bfb6c93a9a8aefec38daab30e475538aeefdfc"}
Jul 20 16:28:42.713515 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:42.713465    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw" podStartSLOduration=1.713444124 podStartE2EDuration="1.713444124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:28:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:28:42.712742937 +0000 UTC m=+2287.368006213" watchObservedRunningTime="2026-07-20 16:28:42.713444124 +0000 UTC m=+2287.368707403"
Jul 20 16:28:43.702935 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:43.702899    2564 generic.go:356] "Generic (PLEG): container finished" podID="3644dd4f-4a47-4d47-9f3a-9521ce6f66f4" containerID="22c76cccb4e14784ae78e9053079e8312170a07128fca2e5c1543a420f2cf4be" exitCode=0
Jul 20 16:28:43.703350 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:43.702953    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw" event={"ID":"3644dd4f-4a47-4d47-9f3a-9521ce6f66f4","Type":"ContainerDied","Data":"22c76cccb4e14784ae78e9053079e8312170a07128fca2e5c1543a420f2cf4be"}
Jul 20 16:28:44.839520 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:44.839493    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"
Jul 20 16:28:44.938944 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:44.938904    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3644dd4f-4a47-4d47-9f3a-9521ce6f66f4-maas-api-service-ca\") pod \"3644dd4f-4a47-4d47-9f3a-9521ce6f66f4\" (UID: \"3644dd4f-4a47-4d47-9f3a-9521ce6f66f4\") "
Jul 20 16:28:44.939310 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:44.939281    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3644dd4f-4a47-4d47-9f3a-9521ce6f66f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3644dd4f-4a47-4d47-9f3a-9521ce6f66f4" (UID: "3644dd4f-4a47-4d47-9f3a-9521ce6f66f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:28:45.040016 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:45.039977    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3644dd4f-4a47-4d47-9f3a-9521ce6f66f4-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:28:45.713023 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:45.712995    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"
Jul 20 16:28:45.713203 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:45.713019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw" event={"ID":"3644dd4f-4a47-4d47-9f3a-9521ce6f66f4","Type":"ContainerDied","Data":"ec86f3f878693771a7b8771b91bfb6c93a9a8aefec38daab30e475538aeefdfc"}
Jul 20 16:28:45.713203 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:45.713064    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec86f3f878693771a7b8771b91bfb6c93a9a8aefec38daab30e475538aeefdfc"
Jul 20 16:28:52.052705 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.052651    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"]
Jul 20 16:28:52.053625 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.053562    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf" podUID="68b5f65b-1264-4eb1-9f22-01cf3f485485" containerName="maas-api" containerID="cri-o://795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35" gracePeriod=30
Jul 20 16:28:52.081640 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.081589    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:52.518069 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.518040    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:28:52.614310 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.614226    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b5f65b-1264-4eb1-9f22-01cf3f485485-maas-api-tls\") pod \"68b5f65b-1264-4eb1-9f22-01cf3f485485\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") "
Jul 20 16:28:52.614310 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.614275    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlwcg\" (UniqueName: \"kubernetes.io/projected/68b5f65b-1264-4eb1-9f22-01cf3f485485-kube-api-access-tlwcg\") pod \"68b5f65b-1264-4eb1-9f22-01cf3f485485\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") "
Jul 20 16:28:52.614310 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.614309    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b5f65b-1264-4eb1-9f22-01cf3f485485-openshift-service-ca\") pod \"68b5f65b-1264-4eb1-9f22-01cf3f485485\" (UID: \"68b5f65b-1264-4eb1-9f22-01cf3f485485\") "
Jul 20 16:28:52.614782 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.614751    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b5f65b-1264-4eb1-9f22-01cf3f485485-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68b5f65b-1264-4eb1-9f22-01cf3f485485" (UID: "68b5f65b-1264-4eb1-9f22-01cf3f485485"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:28:52.616473 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.616446    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68b5f65b-1264-4eb1-9f22-01cf3f485485-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68b5f65b-1264-4eb1-9f22-01cf3f485485" (UID: "68b5f65b-1264-4eb1-9f22-01cf3f485485"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:28:52.616578 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.616492    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68b5f65b-1264-4eb1-9f22-01cf3f485485-kube-api-access-tlwcg" (OuterVolumeSpecName: "kube-api-access-tlwcg") pod "68b5f65b-1264-4eb1-9f22-01cf3f485485" (UID: "68b5f65b-1264-4eb1-9f22-01cf3f485485"). InnerVolumeSpecName "kube-api-access-tlwcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:28:52.715708 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.715648    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b5f65b-1264-4eb1-9f22-01cf3f485485-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:28:52.715708 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.715703    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlwcg\" (UniqueName: \"kubernetes.io/projected/68b5f65b-1264-4eb1-9f22-01cf3f485485-kube-api-access-tlwcg\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:28:52.715920 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.715719    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b5f65b-1264-4eb1-9f22-01cf3f485485-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:28:52.745519 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.745484    2564 generic.go:356] "Generic (PLEG): container finished" podID="68b5f65b-1264-4eb1-9f22-01cf3f485485" containerID="795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35" exitCode=0
Jul 20 16:28:52.745740 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.745544    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"
Jul 20 16:28:52.745740 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.745576    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf" event={"ID":"68b5f65b-1264-4eb1-9f22-01cf3f485485","Type":"ContainerDied","Data":"795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35"}
Jul 20 16:28:52.745740 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.745661    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf" event={"ID":"68b5f65b-1264-4eb1-9f22-01cf3f485485","Type":"ContainerDied","Data":"3d541bbd0406ed185583c1b5818b8efb893c9a00aec24bcb7534dad2d87a43f7"}
Jul 20 16:28:52.745740 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.745718    2564 scope.go:117] "RemoveContainer" containerID="795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35"
Jul 20 16:28:52.754156 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.754137    2564 scope.go:117] "RemoveContainer" containerID="795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35"
Jul 20 16:28:52.754428 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:28:52.754410    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35\": container with ID starting with 795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35 not found: ID does not exist" containerID="795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35"
Jul 20 16:28:52.754480 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.754435    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35"} err="failed to get container status \"795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35\": rpc error: code = NotFound desc = could not find container \"795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35\": container with ID starting with 795dba5ccb4385edef36bdffc5b000a94b82dfc29eebe8a8698ee30b78625b35 not found: ID does not exist"
Jul 20 16:28:52.767708 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.767650    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"]
Jul 20 16:28:52.772086 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:52.772058    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18d8b0-5cb7b5cb5c-kwcgf"]
Jul 20 16:28:53.868570 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:53.868537    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68b5f65b-1264-4eb1-9f22-01cf3f485485" path="/var/lib/kubelet/pods/68b5f65b-1264-4eb1-9f22-01cf3f485485/volumes"
Jul 20 16:28:55.268738 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:55.268707    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:56.617431 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:56.617396    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:56.625251 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:56.625216    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"]
Jul 20 16:28:56.627891 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:56.627854    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-18d8b0-gzztw"]
Jul 20 16:28:57.022917 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:57.022890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:28:57.868888 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:28:57.868856    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3644dd4f-4a47-4d47-9f3a-9521ce6f66f4" path="/var/lib/kubelet/pods/3644dd4f-4a47-4d47-9f3a-9521ce6f66f4/volumes"
Jul 20 16:29:03.053334 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:03.053298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:03.112048 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:03.112012    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:03.326541 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:03.326453    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:03.612396 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:03.612314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:03.910566 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:03.910475    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:04.253715 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:04.253599    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:04.436618 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:04.436586    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:07.194522 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.194479    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:07.281466 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.281426    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:07.282100 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.282080    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"]
Jul 20 16:29:07.282399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.282386    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3644dd4f-4a47-4d47-9f3a-9521ce6f66f4" containerName="revoke-keys"
Jul 20 16:29:07.282461 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.282400    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3644dd4f-4a47-4d47-9f3a-9521ce6f66f4" containerName="revoke-keys"
Jul 20 16:29:07.282461 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.282415    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68b5f65b-1264-4eb1-9f22-01cf3f485485" containerName="maas-api"
Jul 20 16:29:07.282461 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.282422    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="68b5f65b-1264-4eb1-9f22-01cf3f485485" containerName="maas-api"
Jul 20 16:29:07.282622 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.282507    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="68b5f65b-1264-4eb1-9f22-01cf3f485485" containerName="maas-api"
Jul 20 16:29:07.282622 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.282518    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3644dd4f-4a47-4d47-9f3a-9521ce6f66f4" containerName="revoke-keys"
Jul 20 16:29:07.286593 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.286572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.288871 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.288848    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c4f6bb-serving-cert\""
Jul 20 16:29:07.296118 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.296091    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"]
Jul 20 16:29:07.340012 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.339978    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh54r\" (UniqueName: \"kubernetes.io/projected/720e4482-7afb-4697-96b3-75c52cb3cdcb-kube-api-access-dh54r\") pod \"maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.340227 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.340077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720e4482-7afb-4697-96b3-75c52cb3cdcb-maas-api-tls\") pod \"maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.340227 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.340116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/720e4482-7afb-4697-96b3-75c52cb3cdcb-openshift-service-ca\") pod \"maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.344124 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.344086    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:07.441391 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.441353    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh54r\" (UniqueName: \"kubernetes.io/projected/720e4482-7afb-4697-96b3-75c52cb3cdcb-kube-api-access-dh54r\") pod \"maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.441549 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.441445    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720e4482-7afb-4697-96b3-75c52cb3cdcb-maas-api-tls\") pod \"maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.441549 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.441482    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/720e4482-7afb-4697-96b3-75c52cb3cdcb-openshift-service-ca\") pod \"maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.441653 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:29:07.441589    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-serving-cert: secret "maas-api-e2e-api-a-c4f6bb-serving-cert" not found
Jul 20 16:29:07.441712 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:29:07.441673    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/720e4482-7afb-4697-96b3-75c52cb3cdcb-maas-api-tls podName:720e4482-7afb-4697-96b3-75c52cb3cdcb nodeName:}" failed. No retries permitted until 2026-07-20 16:29:07.941653325 +0000 UTC m=+2312.596916592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/720e4482-7afb-4697-96b3-75c52cb3cdcb-maas-api-tls") pod "maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh" (UID: "720e4482-7afb-4697-96b3-75c52cb3cdcb") : secret "maas-api-e2e-api-a-c4f6bb-serving-cert" not found
Jul 20 16:29:07.442226 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.442206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/720e4482-7afb-4697-96b3-75c52cb3cdcb-openshift-service-ca\") pod \"maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.451172 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.451102    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh54r\" (UniqueName: \"kubernetes.io/projected/720e4482-7afb-4697-96b3-75c52cb3cdcb-kube-api-access-dh54r\") pod \"maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.554850 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.554816    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:07.946313 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.946278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720e4482-7afb-4697-96b3-75c52cb3cdcb-maas-api-tls\") pod \"maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.948872 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.948847    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720e4482-7afb-4697-96b3-75c52cb3cdcb-maas-api-tls\") pod \"maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:07.986062 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:07.986026    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:08.087859 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:08.087810    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:08.199093 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:08.199013    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:08.358902 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:08.358875    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"]
Jul 20 16:29:08.362230 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:29:08.362194    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720e4482_7afb_4697_96b3_75c52cb3cdcb.slice/crio-dba3c1e49ce9a8a0016c63ccbad08a242a9c83214bf660a22b1bfe0ad0269360 WatchSource:0}: Error finding container dba3c1e49ce9a8a0016c63ccbad08a242a9c83214bf660a22b1bfe0ad0269360: Status 404 returned error can't find the container with id dba3c1e49ce9a8a0016c63ccbad08a242a9c83214bf660a22b1bfe0ad0269360
Jul 20 16:29:08.819630 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:08.819594    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh" event={"ID":"720e4482-7afb-4697-96b3-75c52cb3cdcb","Type":"ContainerStarted","Data":"dba3c1e49ce9a8a0016c63ccbad08a242a9c83214bf660a22b1bfe0ad0269360"}
Jul 20 16:29:09.825718 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:09.825668    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh" event={"ID":"720e4482-7afb-4697-96b3-75c52cb3cdcb","Type":"ContainerStarted","Data":"03e083174171bffc2a85db7434859f8303d6acad5672279cf1ed16d6187bd83f"}
Jul 20 16:29:09.826232 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:09.825789    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:09.844163 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:09.844105    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh" podStartSLOduration=2.210379735 podStartE2EDuration="2.844086915s" totalImagesPullingTime="633.70718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:29:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:29:09.842553489 +0000 UTC m=+2314.497816777" watchObservedRunningTime="2026-07-20 16:29:09.844086915 +0000 UTC m=+2314.499350241"
Jul 20 16:29:11.678109 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:11.678073    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:12.162569 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:12.162538    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:12.525916 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:12.525880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:13.140674 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:13.140635    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:15.204231 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.204181    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:15.304189 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.304157    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:15.304558 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.304530    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"]
Jul 20 16:29:15.308705 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.308666    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:15.311279 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.311252    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-af2b09-serving-cert\""
Jul 20 16:29:15.327860 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.327823    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"]
Jul 20 16:29:15.399519 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.399482    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:15.415659 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.415617    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtxh5\" (UniqueName: \"kubernetes.io/projected/3c749693-c6a0-4a43-a294-89ffc155acd4-kube-api-access-gtxh5\") pod \"maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:15.415848 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.415764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c749693-c6a0-4a43-a294-89ffc155acd4-openshift-service-ca\") pod \"maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:15.415848 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.415787    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749693-c6a0-4a43-a294-89ffc155acd4-maas-api-tls\") pod \"maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:15.517211 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.517105    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c749693-c6a0-4a43-a294-89ffc155acd4-openshift-service-ca\") pod \"maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:15.517211 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.517152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749693-c6a0-4a43-a294-89ffc155acd4-maas-api-tls\") pod \"maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:15.517211 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.517195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtxh5\" (UniqueName: \"kubernetes.io/projected/3c749693-c6a0-4a43-a294-89ffc155acd4-kube-api-access-gtxh5\") pod \"maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:15.517506 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:29:15.517325    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-serving-cert: secret "maas-api-e2e-api-b-af2b09-serving-cert" not found
Jul 20 16:29:15.517506 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:29:15.517414    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c749693-c6a0-4a43-a294-89ffc155acd4-maas-api-tls podName:3c749693-c6a0-4a43-a294-89ffc155acd4 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:16.017391086 +0000 UTC m=+2320.672654363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c749693-c6a0-4a43-a294-89ffc155acd4-maas-api-tls") pod "maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw" (UID: "3c749693-c6a0-4a43-a294-89ffc155acd4") : secret "maas-api-e2e-api-b-af2b09-serving-cert" not found
Jul 20 16:29:15.517894 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.517875    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c749693-c6a0-4a43-a294-89ffc155acd4-openshift-service-ca\") pod \"maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:15.526277 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.526250    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtxh5\" (UniqueName: \"kubernetes.io/projected/3c749693-c6a0-4a43-a294-89ffc155acd4-kube-api-access-gtxh5\") pod \"maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:15.638618 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.638584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:15.835420 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:15.835388    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:16.005966 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:16.005935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:16.021889 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:16.021851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749693-c6a0-4a43-a294-89ffc155acd4-maas-api-tls\") pod \"maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:16.024430 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:16.024405    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749693-c6a0-4a43-a294-89ffc155acd4-maas-api-tls\") pod \"maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:16.109926 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:16.109831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:16.221387 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:16.221348    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:16.386657 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:16.386535    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"]
Jul 20 16:29:16.389372 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:29:16.389342    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c749693_c6a0_4a43_a294_89ffc155acd4.slice/crio-444c447b7666efa2107eb19a98a5b69afccf88a98e0c72613a84f32071e262bc WatchSource:0}: Error finding container 444c447b7666efa2107eb19a98a5b69afccf88a98e0c72613a84f32071e262bc: Status 404 returned error can't find the container with id 444c447b7666efa2107eb19a98a5b69afccf88a98e0c72613a84f32071e262bc
Jul 20 16:29:16.858789 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:16.858746    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw" event={"ID":"3c749693-c6a0-4a43-a294-89ffc155acd4","Type":"ContainerStarted","Data":"1998b51f75a82507688a5d1f9c982ae8b738c1d3202cea4cd5058d593213da19"}
Jul 20 16:29:16.858941 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:16.858809    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw" event={"ID":"3c749693-c6a0-4a43-a294-89ffc155acd4","Type":"ContainerStarted","Data":"444c447b7666efa2107eb19a98a5b69afccf88a98e0c72613a84f32071e262bc"}
Jul 20 16:29:16.858941 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:16.858836    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:16.877030 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:16.876970    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw" podStartSLOduration=1.544734923 podStartE2EDuration="1.876952518s" totalImagesPullingTime="332.217595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:29:16.875599138 +0000 UTC m=+2321.530862412" watchObservedRunningTime="2026-07-20 16:29:16.876952518 +0000 UTC m=+2321.532215794"
Jul 20 16:29:17.576540 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:17.576503    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:19.719740 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:19.719698    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:22.869411 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:22.869378    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:29:27.769122 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:27.769086    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"]
Jul 20 16:29:27.772876 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:27.772856    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"
Jul 20 16:29:27.775150 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:27.775122    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:29:27.781550 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:27.781486    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"]
Jul 20 16:29:27.827442 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:27.827402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a9b906d-604e-4bdd-aa5e-842f9bb04dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk\" (UID: \"2a9b906d-604e-4bdd-aa5e-842f9bb04dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"
Jul 20 16:29:27.928344 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:27.928309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a9b906d-604e-4bdd-aa5e-842f9bb04dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk\" (UID: \"2a9b906d-604e-4bdd-aa5e-842f9bb04dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"
Jul 20 16:29:27.929081 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:27.929058    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a9b906d-604e-4bdd-aa5e-842f9bb04dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk\" (UID: \"2a9b906d-604e-4bdd-aa5e-842f9bb04dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"
Jul 20 16:29:28.084063 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:28.083984    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"
Jul 20 16:29:28.235599 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:28.235454    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"]
Jul 20 16:29:28.237757 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:29:28.237731    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a9b906d_604e_4bdd_aa5e_842f9bb04dd7.slice/crio-5733ef91232469773d57260f2c2da284105e349712e27f2c98754f8929002657 WatchSource:0}: Error finding container 5733ef91232469773d57260f2c2da284105e349712e27f2c98754f8929002657: Status 404 returned error can't find the container with id 5733ef91232469773d57260f2c2da284105e349712e27f2c98754f8929002657
Jul 20 16:29:28.566181 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:28.566148    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:28.661024 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:28.660987    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:28.914267 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:28.914228    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk" event={"ID":"2a9b906d-604e-4bdd-aa5e-842f9bb04dd7","Type":"ContainerStarted","Data":"84521e334ac86e5b934a04266e930a32f1571654f0a74cc84a69212f6d0c48bd"}
Jul 20 16:29:28.914733 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:28.914297    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk" event={"ID":"2a9b906d-604e-4bdd-aa5e-842f9bb04dd7","Type":"ContainerStarted","Data":"5733ef91232469773d57260f2c2da284105e349712e27f2c98754f8929002657"}
Jul 20 16:29:28.935093 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:28.935041    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk" podStartSLOduration=1.935021698 podStartE2EDuration="1.935021698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:29:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:29:28.93298619 +0000 UTC m=+2333.588249588" watchObservedRunningTime="2026-07-20 16:29:28.935021698 +0000 UTC m=+2333.590284974"
Jul 20 16:29:29.920551 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:29.920514    2564 generic.go:356] "Generic (PLEG): container finished" podID="2a9b906d-604e-4bdd-aa5e-842f9bb04dd7" containerID="84521e334ac86e5b934a04266e930a32f1571654f0a74cc84a69212f6d0c48bd" exitCode=0
Jul 20 16:29:29.920991 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:29.920568    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk" event={"ID":"2a9b906d-604e-4bdd-aa5e-842f9bb04dd7","Type":"ContainerDied","Data":"84521e334ac86e5b934a04266e930a32f1571654f0a74cc84a69212f6d0c48bd"}
Jul 20 16:29:31.062115 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:31.062086    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"
Jul 20 16:29:31.154863 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:31.154822    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a9b906d-604e-4bdd-aa5e-842f9bb04dd7-maas-api-service-ca\") pod \"2a9b906d-604e-4bdd-aa5e-842f9bb04dd7\" (UID: \"2a9b906d-604e-4bdd-aa5e-842f9bb04dd7\") "
Jul 20 16:29:31.155204 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:31.155179    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a9b906d-604e-4bdd-aa5e-842f9bb04dd7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2a9b906d-604e-4bdd-aa5e-842f9bb04dd7" (UID: "2a9b906d-604e-4bdd-aa5e-842f9bb04dd7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:29:31.255671 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:31.255576    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a9b906d-604e-4bdd-aa5e-842f9bb04dd7-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:29:31.931360 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:31.931324    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"
Jul 20 16:29:31.931360 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:31.931352    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk" event={"ID":"2a9b906d-604e-4bdd-aa5e-842f9bb04dd7","Type":"ContainerDied","Data":"5733ef91232469773d57260f2c2da284105e349712e27f2c98754f8929002657"}
Jul 20 16:29:31.931583 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:31.931403    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5733ef91232469773d57260f2c2da284105e349712e27f2c98754f8929002657"
Jul 20 16:29:36.084436 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:36.084406    2564 scope.go:117] "RemoveContainer" containerID="0fc0163403324d7fee17b7c90adfb0fc160a0b8e6e2ae02b43ec05ab1f3a040e"
Jul 20 16:29:36.825904 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:36.825873    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:36.930750 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:36.930715    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:38.543469 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:38.543423    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:38.644921 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:38.644889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:43.816399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:43.816366    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"]
Jul 20 16:29:43.816812 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:43.816591    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh" podUID="720e4482-7afb-4697-96b3-75c52cb3cdcb" containerName="maas-api" containerID="cri-o://03e083174171bffc2a85db7434859f8303d6acad5672279cf1ed16d6187bd83f" gracePeriod=30
Jul 20 16:29:43.831706 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:43.831662    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:43.986795 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:43.986763    2564 generic.go:356] "Generic (PLEG): container finished" podID="720e4482-7afb-4697-96b3-75c52cb3cdcb" containerID="03e083174171bffc2a85db7434859f8303d6acad5672279cf1ed16d6187bd83f" exitCode=0
Jul 20 16:29:43.986977 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:43.986836    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh" event={"ID":"720e4482-7afb-4697-96b3-75c52cb3cdcb","Type":"ContainerDied","Data":"03e083174171bffc2a85db7434859f8303d6acad5672279cf1ed16d6187bd83f"}
Jul 20 16:29:44.068390 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.068310    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:44.169193 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.169164    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh54r\" (UniqueName: \"kubernetes.io/projected/720e4482-7afb-4697-96b3-75c52cb3cdcb-kube-api-access-dh54r\") pod \"720e4482-7afb-4697-96b3-75c52cb3cdcb\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") "
Jul 20 16:29:44.169428 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.169238    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/720e4482-7afb-4697-96b3-75c52cb3cdcb-openshift-service-ca\") pod \"720e4482-7afb-4697-96b3-75c52cb3cdcb\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") "
Jul 20 16:29:44.169428 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.169279    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720e4482-7afb-4697-96b3-75c52cb3cdcb-maas-api-tls\") pod \"720e4482-7afb-4697-96b3-75c52cb3cdcb\" (UID: \"720e4482-7afb-4697-96b3-75c52cb3cdcb\") "
Jul 20 16:29:44.169676 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.169649    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/720e4482-7afb-4697-96b3-75c52cb3cdcb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "720e4482-7afb-4697-96b3-75c52cb3cdcb" (UID: "720e4482-7afb-4697-96b3-75c52cb3cdcb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:29:44.171374 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.171352    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/720e4482-7afb-4697-96b3-75c52cb3cdcb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "720e4482-7afb-4697-96b3-75c52cb3cdcb" (UID: "720e4482-7afb-4697-96b3-75c52cb3cdcb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:29:44.171485 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.171462    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/720e4482-7afb-4697-96b3-75c52cb3cdcb-kube-api-access-dh54r" (OuterVolumeSpecName: "kube-api-access-dh54r") pod "720e4482-7afb-4697-96b3-75c52cb3cdcb" (UID: "720e4482-7afb-4697-96b3-75c52cb3cdcb"). InnerVolumeSpecName "kube-api-access-dh54r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:29:44.270399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.270360    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh54r\" (UniqueName: \"kubernetes.io/projected/720e4482-7afb-4697-96b3-75c52cb3cdcb-kube-api-access-dh54r\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:29:44.270399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.270393    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/720e4482-7afb-4697-96b3-75c52cb3cdcb-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:29:44.270399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.270409    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720e4482-7afb-4697-96b3-75c52cb3cdcb-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:29:44.998705 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.998667    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"
Jul 20 16:29:44.999122 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.998668    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh" event={"ID":"720e4482-7afb-4697-96b3-75c52cb3cdcb","Type":"ContainerDied","Data":"dba3c1e49ce9a8a0016c63ccbad08a242a9c83214bf660a22b1bfe0ad0269360"}
Jul 20 16:29:44.999122 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:44.998837    2564 scope.go:117] "RemoveContainer" containerID="03e083174171bffc2a85db7434859f8303d6acad5672279cf1ed16d6187bd83f"
Jul 20 16:29:45.020985 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:45.020947    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"]
Jul 20 16:29:45.026491 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:45.026464    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4f6bb-6b7ff664d7-bwgrh"]
Jul 20 16:29:45.869696 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:45.869652    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="720e4482-7afb-4697-96b3-75c52cb3cdcb" path="/var/lib/kubelet/pods/720e4482-7afb-4697-96b3-75c52cb3cdcb/volumes"
Jul 20 16:29:47.855704 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:47.855628    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"]
Jul 20 16:29:47.858091 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:47.858061    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c4f6bb-g9vvk"]
Jul 20 16:29:47.858287 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:47.858265    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:47.869349 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:47.869322    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a9b906d-604e-4bdd-aa5e-842f9bb04dd7" path="/var/lib/kubelet/pods/2a9b906d-604e-4bdd-aa5e-842f9bb04dd7/volumes"
Jul 20 16:29:48.255139 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:48.255062    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:48.896074 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:48.896043    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:48.996196 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:48.996164    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:54.780475 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:54.780442    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:54.833896 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:54.833863    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:55.085263 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.085171    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:55.305541 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.305511    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:55.432311 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.432221    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"]
Jul 20 16:29:55.432773 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.432754    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720e4482-7afb-4697-96b3-75c52cb3cdcb" containerName="maas-api"
Jul 20 16:29:55.432827 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.432777    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="720e4482-7afb-4697-96b3-75c52cb3cdcb" containerName="maas-api"
Jul 20 16:29:55.432827 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.432789    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a9b906d-604e-4bdd-aa5e-842f9bb04dd7" containerName="revoke-keys"
Jul 20 16:29:55.432827 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.432797    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a9b906d-604e-4bdd-aa5e-842f9bb04dd7" containerName="revoke-keys"
Jul 20 16:29:55.432925 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.432898    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a9b906d-604e-4bdd-aa5e-842f9bb04dd7" containerName="revoke-keys"
Jul 20 16:29:55.432925 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.432910    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="720e4482-7afb-4697-96b3-75c52cb3cdcb" containerName="maas-api"
Jul 20 16:29:55.437362 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.437340    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"
Jul 20 16:29:55.439523 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.439498    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:29:55.445797 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.445770    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"]
Jul 20 16:29:55.475575 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.475540    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db1b1f0-4eb7-476f-8362-e4b57170f271-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm\" (UID: \"2db1b1f0-4eb7-476f-8362-e4b57170f271\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"
Jul 20 16:29:55.576473 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.576427    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db1b1f0-4eb7-476f-8362-e4b57170f271-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm\" (UID: \"2db1b1f0-4eb7-476f-8362-e4b57170f271\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"
Jul 20 16:29:55.577109 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.577085    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db1b1f0-4eb7-476f-8362-e4b57170f271-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm\" (UID: \"2db1b1f0-4eb7-476f-8362-e4b57170f271\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"
Jul 20 16:29:55.748893 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.748802    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"
Jul 20 16:29:55.890736 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:55.890697    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"]
Jul 20 16:29:55.893504 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:29:55.893475    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db1b1f0_4eb7_476f_8362_e4b57170f271.slice/crio-688a31c70ec389c35239f27bf34a5d9c09ba9905a517552dd6af7bddc5df809e WatchSource:0}: Error finding container 688a31c70ec389c35239f27bf34a5d9c09ba9905a517552dd6af7bddc5df809e: Status 404 returned error can't find the container with id 688a31c70ec389c35239f27bf34a5d9c09ba9905a517552dd6af7bddc5df809e
Jul 20 16:29:56.055678 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:56.055644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm" event={"ID":"2db1b1f0-4eb7-476f-8362-e4b57170f271","Type":"ContainerStarted","Data":"688a31c70ec389c35239f27bf34a5d9c09ba9905a517552dd6af7bddc5df809e"}
Jul 20 16:29:56.111036 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:56.111005    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:56.211352 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:56.211303    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:29:57.061633 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:57.061598    2564 generic.go:356] "Generic (PLEG): container finished" podID="2db1b1f0-4eb7-476f-8362-e4b57170f271" containerID="c0b14662875b9eea6a2aae7b08bdac2e1f2dc0398f6e3ddf6226e8855973042f" exitCode=0
Jul 20 16:29:57.062063 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:57.061647    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm" event={"ID":"2db1b1f0-4eb7-476f-8362-e4b57170f271","Type":"ContainerDied","Data":"c0b14662875b9eea6a2aae7b08bdac2e1f2dc0398f6e3ddf6226e8855973042f"}
Jul 20 16:29:58.224605 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:58.224569    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"
Jul 20 16:29:58.398501 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:58.398412    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db1b1f0-4eb7-476f-8362-e4b57170f271-maas-api-service-ca\") pod \"2db1b1f0-4eb7-476f-8362-e4b57170f271\" (UID: \"2db1b1f0-4eb7-476f-8362-e4b57170f271\") "
Jul 20 16:29:58.398808 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:58.398785    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db1b1f0-4eb7-476f-8362-e4b57170f271-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2db1b1f0-4eb7-476f-8362-e4b57170f271" (UID: "2db1b1f0-4eb7-476f-8362-e4b57170f271"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:29:58.499636 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:58.499601    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db1b1f0-4eb7-476f-8362-e4b57170f271-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:29:59.072222 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:59.072193    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"
Jul 20 16:29:59.072222 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:59.072210    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm" event={"ID":"2db1b1f0-4eb7-476f-8362-e4b57170f271","Type":"ContainerDied","Data":"688a31c70ec389c35239f27bf34a5d9c09ba9905a517552dd6af7bddc5df809e"}
Jul 20 16:29:59.072477 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:29:59.072257    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="688a31c70ec389c35239f27bf34a5d9c09ba9905a517552dd6af7bddc5df809e"
Jul 20 16:30:00.199000 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.198956    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq"]
Jul 20 16:30:00.199519 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.199302    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2db1b1f0-4eb7-476f-8362-e4b57170f271" containerName="revoke-keys"
Jul 20 16:30:00.199519 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.199315    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2db1b1f0-4eb7-476f-8362-e4b57170f271" containerName="revoke-keys"
Jul 20 16:30:00.199519 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.199398    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2db1b1f0-4eb7-476f-8362-e4b57170f271" containerName="revoke-keys"
Jul 20 16:30:00.202378 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.202357    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs"]
Jul 20 16:30:00.202517 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.202488    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq"
Jul 20 16:30:00.205420 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.205402    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs"
Jul 20 16:30:00.206433 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.206412    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:30:00.213019 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.212994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq"
Jul 20 16:30:00.215954 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.215896    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs"]
Jul 20 16:30:00.218236 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.218214    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq"]
Jul 20 16:30:00.225033 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.225005    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs"
Jul 20 16:30:00.371623 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.371579    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq"]
Jul 20 16:30:00.377352 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:30:00.377311    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce871b0_0fec_42a9_a45b_eae92687c2c8.slice/crio-57d0fbdf35dc0496a09a476e99a5cdd64d6299c5e09a66596e830123b049fffc WatchSource:0}: Error finding container 57d0fbdf35dc0496a09a476e99a5cdd64d6299c5e09a66596e830123b049fffc: Status 404 returned error can't find the container with id 57d0fbdf35dc0496a09a476e99a5cdd64d6299c5e09a66596e830123b049fffc
Jul 20 16:30:00.388035 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:00.388007    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs"]
Jul 20 16:30:00.392164 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:30:00.392134    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080b0fc6_cbfc_45ff_8875_7a7e0393ec04.slice/crio-d1276d894c998d89dc26a6087cd103ed0420f20f4622112d4ed5a1468c054f7f WatchSource:0}: Error finding container d1276d894c998d89dc26a6087cd103ed0420f20f4622112d4ed5a1468c054f7f: Status 404 returned error can't find the container with id d1276d894c998d89dc26a6087cd103ed0420f20f4622112d4ed5a1468c054f7f
Jul 20 16:30:01.082744 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:01.082702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq" event={"ID":"4ce871b0-0fec-42a9-a45b-eae92687c2c8","Type":"ContainerStarted","Data":"091e44abbff97981ce857310379a4525d584e03efe19c6d154c2e61ea58f20b9"}
Jul 20 16:30:01.082957 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:01.082762    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq" event={"ID":"4ce871b0-0fec-42a9-a45b-eae92687c2c8","Type":"ContainerStarted","Data":"57d0fbdf35dc0496a09a476e99a5cdd64d6299c5e09a66596e830123b049fffc"}
Jul 20 16:30:01.084319 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:01.084282    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs" event={"ID":"080b0fc6-cbfc-45ff-8875-7a7e0393ec04","Type":"ContainerStarted","Data":"9fc370a7b6c05b27f9f78409c186d6c8ae4f87adbcbf0d2e3be94495af5d609c"}
Jul 20 16:30:01.084441 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:01.084325    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs" event={"ID":"080b0fc6-cbfc-45ff-8875-7a7e0393ec04","Type":"ContainerStarted","Data":"d1276d894c998d89dc26a6087cd103ed0420f20f4622112d4ed5a1468c054f7f"}
Jul 20 16:30:01.099417 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:01.099365    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq" podStartSLOduration=1.099347278 podStartE2EDuration="1.099347278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:01.098023791 +0000 UTC m=+2365.753287069" watchObservedRunningTime="2026-07-20 16:30:01.099347278 +0000 UTC m=+2365.754610554"
Jul 20 16:30:01.111872 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:01.111817    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs" podStartSLOduration=1.111795099 podStartE2EDuration="1.111795099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:01.111243108 +0000 UTC m=+2365.766506388" watchObservedRunningTime="2026-07-20 16:30:01.111795099 +0000 UTC m=+2365.767058376"
Jul 20 16:30:02.090823 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:02.090789    2564 generic.go:356] "Generic (PLEG): container finished" podID="080b0fc6-cbfc-45ff-8875-7a7e0393ec04" containerID="9fc370a7b6c05b27f9f78409c186d6c8ae4f87adbcbf0d2e3be94495af5d609c" exitCode=0
Jul 20 16:30:02.091305 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:02.090874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs" event={"ID":"080b0fc6-cbfc-45ff-8875-7a7e0393ec04","Type":"ContainerDied","Data":"9fc370a7b6c05b27f9f78409c186d6c8ae4f87adbcbf0d2e3be94495af5d609c"}
Jul 20 16:30:02.092401 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:02.092376    2564 generic.go:356] "Generic (PLEG): container finished" podID="4ce871b0-0fec-42a9-a45b-eae92687c2c8" containerID="091e44abbff97981ce857310379a4525d584e03efe19c6d154c2e61ea58f20b9" exitCode=0
Jul 20 16:30:02.092514 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:02.092438    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq" event={"ID":"4ce871b0-0fec-42a9-a45b-eae92687c2c8","Type":"ContainerDied","Data":"091e44abbff97981ce857310379a4525d584e03efe19c6d154c2e61ea58f20b9"}
Jul 20 16:30:03.232485 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:03.232451    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs"
Jul 20 16:30:03.259201 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:03.259170    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq"
Jul 20 16:30:04.103508 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:04.103475    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs"
Jul 20 16:30:04.103733 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:04.103512    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs" event={"ID":"080b0fc6-cbfc-45ff-8875-7a7e0393ec04","Type":"ContainerDied","Data":"d1276d894c998d89dc26a6087cd103ed0420f20f4622112d4ed5a1468c054f7f"}
Jul 20 16:30:04.103733 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:04.103590    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1276d894c998d89dc26a6087cd103ed0420f20f4622112d4ed5a1468c054f7f"
Jul 20 16:30:04.104830 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:04.104803    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq" event={"ID":"4ce871b0-0fec-42a9-a45b-eae92687c2c8","Type":"ContainerDied","Data":"57d0fbdf35dc0496a09a476e99a5cdd64d6299c5e09a66596e830123b049fffc"}
Jul 20 16:30:04.104950 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:04.104838    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742750-tlbmq"
Jul 20 16:30:04.104950 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:04.104846    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57d0fbdf35dc0496a09a476e99a5cdd64d6299c5e09a66596e830123b049fffc"
Jul 20 16:30:11.005960 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.005915    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"]
Jul 20 16:30:11.006487 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.006218    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw" podUID="3c749693-c6a0-4a43-a294-89ffc155acd4" containerName="maas-api" containerID="cri-o://1998b51f75a82507688a5d1f9c982ae8b738c1d3202cea4cd5058d593213da19" gracePeriod=30
Jul 20 16:30:11.025098 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.025070    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:11.037942 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.037733    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs"]
Jul 20 16:30:11.039871 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.039843    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-af2b09-29742750-bfpvs"]
Jul 20 16:30:11.137631 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.137602    2564 generic.go:356] "Generic (PLEG): container finished" podID="3c749693-c6a0-4a43-a294-89ffc155acd4" containerID="1998b51f75a82507688a5d1f9c982ae8b738c1d3202cea4cd5058d593213da19" exitCode=0
Jul 20 16:30:11.137803 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.137667    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw" event={"ID":"3c749693-c6a0-4a43-a294-89ffc155acd4","Type":"ContainerDied","Data":"1998b51f75a82507688a5d1f9c982ae8b738c1d3202cea4cd5058d593213da19"}
Jul 20 16:30:11.259643 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.259582    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:30:11.293923 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.293896    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c749693-c6a0-4a43-a294-89ffc155acd4-openshift-service-ca\") pod \"3c749693-c6a0-4a43-a294-89ffc155acd4\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") "
Jul 20 16:30:11.294100 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.293976    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtxh5\" (UniqueName: \"kubernetes.io/projected/3c749693-c6a0-4a43-a294-89ffc155acd4-kube-api-access-gtxh5\") pod \"3c749693-c6a0-4a43-a294-89ffc155acd4\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") "
Jul 20 16:30:11.294316 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.294289    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c749693-c6a0-4a43-a294-89ffc155acd4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c749693-c6a0-4a43-a294-89ffc155acd4" (UID: "3c749693-c6a0-4a43-a294-89ffc155acd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:30:11.296077 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.296054    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c749693-c6a0-4a43-a294-89ffc155acd4-kube-api-access-gtxh5" (OuterVolumeSpecName: "kube-api-access-gtxh5") pod "3c749693-c6a0-4a43-a294-89ffc155acd4" (UID: "3c749693-c6a0-4a43-a294-89ffc155acd4"). InnerVolumeSpecName "kube-api-access-gtxh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:30:11.394949 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.394911    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749693-c6a0-4a43-a294-89ffc155acd4-maas-api-tls\") pod \"3c749693-c6a0-4a43-a294-89ffc155acd4\" (UID: \"3c749693-c6a0-4a43-a294-89ffc155acd4\") "
Jul 20 16:30:11.395156 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.395106    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c749693-c6a0-4a43-a294-89ffc155acd4-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:30:11.395156 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.395118    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtxh5\" (UniqueName: \"kubernetes.io/projected/3c749693-c6a0-4a43-a294-89ffc155acd4-kube-api-access-gtxh5\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:30:11.397036 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.397007    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c749693-c6a0-4a43-a294-89ffc155acd4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c749693-c6a0-4a43-a294-89ffc155acd4" (UID: "3c749693-c6a0-4a43-a294-89ffc155acd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:30:11.496346 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.496304    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749693-c6a0-4a43-a294-89ffc155acd4-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:30:11.869319 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:11.869275    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="080b0fc6-cbfc-45ff-8875-7a7e0393ec04" path="/var/lib/kubelet/pods/080b0fc6-cbfc-45ff-8875-7a7e0393ec04/volumes"
Jul 20 16:30:12.144075 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:12.143992    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"
Jul 20 16:30:12.144075 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:12.144007    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw" event={"ID":"3c749693-c6a0-4a43-a294-89ffc155acd4","Type":"ContainerDied","Data":"444c447b7666efa2107eb19a98a5b69afccf88a98e0c72613a84f32071e262bc"}
Jul 20 16:30:12.144075 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:12.144064    2564 scope.go:117] "RemoveContainer" containerID="1998b51f75a82507688a5d1f9c982ae8b738c1d3202cea4cd5058d593213da19"
Jul 20 16:30:12.161512 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:12.161480    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"]
Jul 20 16:30:12.164744 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:12.164714    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2b09-7cc486d955-rf9sw"]
Jul 20 16:30:13.869415 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:13.869380    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c749693-c6a0-4a43-a294-89ffc155acd4" path="/var/lib/kubelet/pods/3c749693-c6a0-4a43-a294-89ffc155acd4/volumes"
Jul 20 16:30:15.510419 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:15.510382    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"]
Jul 20 16:30:15.513155 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:15.513126    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:15.513623 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:15.513605    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af2b09-fgkkm"]
Jul 20 16:30:15.877041 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:15.877002    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2db1b1f0-4eb7-476f-8362-e4b57170f271" path="/var/lib/kubelet/pods/2db1b1f0-4eb7-476f-8362-e4b57170f271/volumes"
Jul 20 16:30:15.911870 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:15.911821    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:21.909155 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:21.909123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:22.009293 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:22.009259    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:22.171779 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:22.171633    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:22.418315 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:22.418284    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:22.823995 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:22.823963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:23.080932 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:23.080839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:23.464798 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:23.464680    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:25.660657 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.660615    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:25.751156 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.751115    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"]
Jul 20 16:30:25.751446 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.751434    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c749693-c6a0-4a43-a294-89ffc155acd4" containerName="maas-api"
Jul 20 16:30:25.751490 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.751448    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c749693-c6a0-4a43-a294-89ffc155acd4" containerName="maas-api"
Jul 20 16:30:25.751490 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.751459    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="080b0fc6-cbfc-45ff-8875-7a7e0393ec04" containerName="cleanup"
Jul 20 16:30:25.751490 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.751465    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="080b0fc6-cbfc-45ff-8875-7a7e0393ec04" containerName="cleanup"
Jul 20 16:30:25.751490 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.751477    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ce871b0-0fec-42a9-a45b-eae92687c2c8" containerName="cleanup"
Jul 20 16:30:25.751490 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.751483    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ce871b0-0fec-42a9-a45b-eae92687c2c8" containerName="cleanup"
Jul 20 16:30:25.751644 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.751540    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="080b0fc6-cbfc-45ff-8875-7a7e0393ec04" containerName="cleanup"
Jul 20 16:30:25.751644 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.751547    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ce871b0-0fec-42a9-a45b-eae92687c2c8" containerName="cleanup"
Jul 20 16:30:25.751644 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.751553    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c749693-c6a0-4a43-a294-89ffc155acd4" containerName="maas-api"
Jul 20 16:30:25.756195 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.756175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:25.759123 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.759102    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-329d3d-serving-cert\""
Jul 20 16:30:25.767065 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.767040    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"]
Jul 20 16:30:25.803015 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.802969    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f63d717-1ca0-4ba9-ad7a-a059b178f164-openshift-service-ca\") pod \"maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:25.803220 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.803019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f63d717-1ca0-4ba9-ad7a-a059b178f164-maas-api-tls\") pod \"maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:25.803220 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.803048    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7ddg\" (UniqueName: \"kubernetes.io/projected/5f63d717-1ca0-4ba9-ad7a-a059b178f164-kube-api-access-z7ddg\") pod \"maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:25.903875 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.903837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f63d717-1ca0-4ba9-ad7a-a059b178f164-openshift-service-ca\") pod \"maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:25.904081 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.903893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f63d717-1ca0-4ba9-ad7a-a059b178f164-maas-api-tls\") pod \"maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:25.904081 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.903925    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7ddg\" (UniqueName: \"kubernetes.io/projected/5f63d717-1ca0-4ba9-ad7a-a059b178f164-kube-api-access-z7ddg\") pod \"maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:25.904081 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:30:25.904045    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-serving-cert: secret "maas-api-e2e-shared-a-329d3d-serving-cert" not found
Jul 20 16:30:25.904256 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:30:25.904123    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f63d717-1ca0-4ba9-ad7a-a059b178f164-maas-api-tls podName:5f63d717-1ca0-4ba9-ad7a-a059b178f164 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:26.404100218 +0000 UTC m=+2391.059363495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f63d717-1ca0-4ba9-ad7a-a059b178f164-maas-api-tls") pod "maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs" (UID: "5f63d717-1ca0-4ba9-ad7a-a059b178f164") : secret "maas-api-e2e-shared-a-329d3d-serving-cert" not found
Jul 20 16:30:25.904554 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.904533    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f63d717-1ca0-4ba9-ad7a-a059b178f164-openshift-service-ca\") pod \"maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:25.911998 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:25.911923    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7ddg\" (UniqueName: \"kubernetes.io/projected/5f63d717-1ca0-4ba9-ad7a-a059b178f164-kube-api-access-z7ddg\") pod \"maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:26.237054 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:26.236947    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:26.337472 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:26.337440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:26.409080 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:26.409031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f63d717-1ca0-4ba9-ad7a-a059b178f164-maas-api-tls\") pod \"maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:26.411829 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:26.411799    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f63d717-1ca0-4ba9-ad7a-a059b178f164-maas-api-tls\") pod \"maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:26.455362 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:26.455310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:26.557066 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:26.557034    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:26.667405 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:26.667365    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:26.813324 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:26.813235    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"]
Jul 20 16:30:26.816930 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:30:26.816898    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f63d717_1ca0_4ba9_ad7a_a059b178f164.slice/crio-68787857662b2e2b2ae5efe3d8966d4b0f641006c2aa2b35f159460401c3bcb8 WatchSource:0}: Error finding container 68787857662b2e2b2ae5efe3d8966d4b0f641006c2aa2b35f159460401c3bcb8: Status 404 returned error can't find the container with id 68787857662b2e2b2ae5efe3d8966d4b0f641006c2aa2b35f159460401c3bcb8
Jul 20 16:30:27.217986 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:27.217957    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs" event={"ID":"5f63d717-1ca0-4ba9-ad7a-a059b178f164","Type":"ContainerStarted","Data":"68787857662b2e2b2ae5efe3d8966d4b0f641006c2aa2b35f159460401c3bcb8"}
Jul 20 16:30:28.224385 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:28.224343    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs" event={"ID":"5f63d717-1ca0-4ba9-ad7a-a059b178f164","Type":"ContainerStarted","Data":"7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28"}
Jul 20 16:30:28.224812 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:28.224446    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:28.267620 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:28.265633    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs" podStartSLOduration=2.908327179 podStartE2EDuration="3.265611536s" totalImagesPullingTime="357.284357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:28.262182809 +0000 UTC m=+2392.917446081" watchObservedRunningTime="2026-07-20 16:30:28.265611536 +0000 UTC m=+2392.920874815"
Jul 20 16:30:30.208670 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:30.208614    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:30.522064 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:30.522026    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:30.836265 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:30.836177    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:31.425842 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:31.425809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:33.742674 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.742612    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:33.834142 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.834109    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:33.838372 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.838342    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"]
Jul 20 16:30:33.842770 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.842749    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:33.845086 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.845064    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-02789d-serving-cert\""
Jul 20 16:30:33.854622 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.854596    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"]
Jul 20 16:30:33.878678 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.878646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-maas-api-tls\") pod \"maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:33.878870 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.878739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6fj9\" (UniqueName: \"kubernetes.io/projected/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-kube-api-access-j6fj9\") pod \"maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:33.878870 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.878778    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:33.892262 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.892233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:33.979753 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.979672    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6fj9\" (UniqueName: \"kubernetes.io/projected/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-kube-api-access-j6fj9\") pod \"maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:33.980037 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.979891    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:33.980037 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.979997    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-maas-api-tls\") pod \"maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:33.980566 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.980544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:33.982395 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.982377    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-maas-api-tls\") pod \"maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:33.987770 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.987733    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6fj9\" (UniqueName: \"kubernetes.io/projected/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-kube-api-access-j6fj9\") pod \"maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:33.993091 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:33.993016    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:34.154276 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:34.154239    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:34.236770 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:34.236738    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:30:34.302207 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:34.302182    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"]
Jul 20 16:30:34.429435 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:34.429397    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:34.528712 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:34.528658    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:35.262622 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:35.262589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4" event={"ID":"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc","Type":"ContainerStarted","Data":"68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e"}
Jul 20 16:30:35.263065 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:35.262638    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4" event={"ID":"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc","Type":"ContainerStarted","Data":"e14ed0b517bba65fe41cfb6ccbcc77aa0537e0b7afca8317bcb1177a4b572407"}
Jul 20 16:30:35.263065 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:35.262727    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:35.279640 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:35.279576    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4" podStartSLOduration=1.934244211 podStartE2EDuration="2.279556073s" totalImagesPullingTime="345.311862ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:35.278796692 +0000 UTC m=+2399.934059971" watchObservedRunningTime="2026-07-20 16:30:35.279556073 +0000 UTC m=+2399.934819352"
Jul 20 16:30:35.940567 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:35.940541    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:30:35.941805 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:35.941784    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:30:36.106126 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:36.106097    2564 scope.go:117] "RemoveContainer" containerID="16650c1180f3de37077ff4a2240a00b1ebdafabaae90f186f4c9dfdf25e06be7"
Jul 20 16:30:38.226131 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:38.226096    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:41.273893 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:41.273864    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:30:47.322863 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:47.322828    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:47.428174 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:47.428114    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:48.790016 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:48.789982    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:55.999763 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:55.999721    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:56.108399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:30:56.108363    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:09.272079 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.271995    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"]
Jul 20 16:31:09.276138 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.276113    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.278213 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.278182    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"openshift-service-ca.crt\""
Jul 20 16:31:09.278343 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.278237    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"kube-root-ca.crt\""
Jul 20 16:31:09.278948 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.278922    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"default-dockercfg-b7wm4\""
Jul 20 16:31:09.278948 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.278945    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"auth-test-model-02789d-kserve-self-signed-certs\""
Jul 20 16:31:09.282479 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.282456    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"]
Jul 20 16:31:09.290870 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.290845    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tmp-dir\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.290995 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.290875    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-dshm\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.290995 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.290902    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tls-certs\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.290995 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.290974    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-home\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.291165 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.291056    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-model-cache\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.291165 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.291095    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qk2v\" (UniqueName: \"kubernetes.io/projected/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-kube-api-access-4qk2v\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.299338 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.299311    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:09.392198 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.392154    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tmp-dir\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.392198 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.392198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-dshm\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.392474 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.392234    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tls-certs\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.392474 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.392258    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-home\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.392474 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.392327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-model-cache\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.392474 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.392359    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qk2v\" (UniqueName: \"kubernetes.io/projected/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-kube-api-access-4qk2v\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.392731 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.392645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tmp-dir\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.392815 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.392739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-home\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.392815 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.392775    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-model-cache\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.394676 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.394655    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-dshm\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.394808 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.394793    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tls-certs\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.399909 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.399887    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qk2v\" (UniqueName: \"kubernetes.io/projected/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-kube-api-access-4qk2v\") pod \"auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") " pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.587079 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.586973    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:09.719950 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:09.719925    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"]
Jul 20 16:31:09.722502 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:31:09.722475    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa5151d2_6f5e_4d26_9ee7_b6ed51ee099e.slice/crio-80201de3227b671c9250e8f6a273e852dfef30a4d255fbefba207b104d55f7f7 WatchSource:0}: Error finding container 80201de3227b671c9250e8f6a273e852dfef30a4d255fbefba207b104d55f7f7: Status 404 returned error can't find the container with id 80201de3227b671c9250e8f6a273e852dfef30a4d255fbefba207b104d55f7f7
Jul 20 16:31:10.429865 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:10.429825    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc" event={"ID":"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e","Type":"ContainerStarted","Data":"89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0"}
Jul 20 16:31:10.429865 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:10.429866    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc" event={"ID":"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e","Type":"ContainerStarted","Data":"80201de3227b671c9250e8f6a273e852dfef30a4d255fbefba207b104d55f7f7"}
Jul 20 16:31:10.446105 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:10.446043    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc" podStartSLOduration=1.446022929 podStartE2EDuration="1.446022929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:31:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:31:10.444461316 +0000 UTC m=+2435.099724592" watchObservedRunningTime="2026-07-20 16:31:10.446022929 +0000 UTC m=+2435.101286207"
Jul 20 16:31:19.587338 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:19.587296    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:19.587850 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:19.587353    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:19.600066 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:19.600038    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:31:20.488673 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:31:20.488639    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:33:15.565243 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:15.565197    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:16.355799 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:16.355761    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"]
Jul 20 16:33:16.374578 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:16.374542    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:25.874198 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:25.874163    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:31.377801 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.377735    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc" podUID="fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" containerName="main" containerID="cri-o://89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0" gracePeriod=45
Jul 20 16:33:31.633946 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.633880    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:33:31.727361 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.727326    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qk2v\" (UniqueName: \"kubernetes.io/projected/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-kube-api-access-4qk2v\") pod \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") "
Jul 20 16:33:31.727539 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.727375    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-dshm\") pod \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") "
Jul 20 16:33:31.727581 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.727534    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-model-cache\") pod \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") "
Jul 20 16:33:31.727617 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.727583    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tls-certs\") pod \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") "
Jul 20 16:33:31.727651 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.727638    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tmp-dir\") pod \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") "
Jul 20 16:33:31.727720 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.727683    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-home\") pod \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\" (UID: \"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e\") "
Jul 20 16:33:31.727914 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.727882    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-model-cache" (OuterVolumeSpecName: "model-cache") pod "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" (UID: "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:33:31.727914 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.727892    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" (UID: "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:33:31.728107 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.728028    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-home" (OuterVolumeSpecName: "home") pod "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" (UID: "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:33:31.728107 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.728052    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tmp-dir\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:33:31.728107 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.728073    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-model-cache\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:33:31.729826 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.729796    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-kube-api-access-4qk2v" (OuterVolumeSpecName: "kube-api-access-4qk2v") pod "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" (UID: "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e"). InnerVolumeSpecName "kube-api-access-4qk2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:33:31.729826 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.729817    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-dshm" (OuterVolumeSpecName: "dshm") pod "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" (UID: "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:33:31.729956 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.729828    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" (UID: "fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:33:31.829419 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.829381    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-tls-certs\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:33:31.829419 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.829417    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-home\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:33:31.829637 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.829434    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qk2v\" (UniqueName: \"kubernetes.io/projected/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-kube-api-access-4qk2v\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:33:31.829637 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:31.829449    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e-dshm\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:33:32.089663 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:32.089625    2564 generic.go:356] "Generic (PLEG): container finished" podID="fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" containerID="89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0" exitCode=0
Jul 20 16:33:32.089866 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:32.089676    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"
Jul 20 16:33:32.089866 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:32.089715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc" event={"ID":"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e","Type":"ContainerDied","Data":"89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0"}
Jul 20 16:33:32.089866 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:32.089761    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc" event={"ID":"fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e","Type":"ContainerDied","Data":"80201de3227b671c9250e8f6a273e852dfef30a4d255fbefba207b104d55f7f7"}
Jul 20 16:33:32.089866 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:32.089779    2564 scope.go:117] "RemoveContainer" containerID="89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0"
Jul 20 16:33:32.098210 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:32.098191    2564 scope.go:117] "RemoveContainer" containerID="89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0"
Jul 20 16:33:32.098484 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:33:32.098465    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0\": container with ID starting with 89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0 not found: ID does not exist" containerID="89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0"
Jul 20 16:33:32.098539 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:32.098492    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0"} err="failed to get container status \"89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0\": rpc error: code = NotFound desc = could not find container \"89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0\": container with ID starting with 89018eefe96bd17662df9c7a0665650c09f6525f240452d45af2459173adf8d0 not found: ID does not exist"
Jul 20 16:33:32.104469 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:32.104446    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"]
Jul 20 16:33:32.108006 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:32.107978    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/auth-test-model-02789d-kserve-6bdc8cf9f5-z9jlc"]
Jul 20 16:33:33.868486 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:33.868451    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" path="/var/lib/kubelet/pods/fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e/volumes"
Jul 20 16:33:46.059580 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.059537    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"]
Jul 20 16:33:46.060083 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.060052    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" containerName="main"
Jul 20 16:33:46.060083 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.060076    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" containerName="main"
Jul 20 16:33:46.060203 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.060191    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa5151d2-6f5e-4d26-9ee7-b6ed51ee099e" containerName="main"
Jul 20 16:33:46.063427 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.063405    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.066012 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.065987    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"default-dockercfg-b7wm4\""
Jul 20 16:33:46.066124 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.066013    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"kube-root-ca.crt\""
Jul 20 16:33:46.066124 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.066049    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"sub-test-model-02789d-kserve-self-signed-certs\""
Jul 20 16:33:46.066247 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.065986    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"openshift-service-ca.crt\""
Jul 20 16:33:46.069661 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.069635    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"]
Jul 20 16:33:46.095000 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.094969    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:46.160322 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.160288    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffb5b\" (UniqueName: \"kubernetes.io/projected/98c3d99d-6fa8-43b3-b76d-3da0abac8968-kube-api-access-ffb5b\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.160510 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.160332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-home\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.160510 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.160375    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tls-certs\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.160510 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.160402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-dshm\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.160510 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.160478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-model-cache\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.160722 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.160544    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tmp-dir\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.261566 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.261530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-dshm\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.261816 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.261586    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-model-cache\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.261816 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.261616    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tmp-dir\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.261816 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.261654    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffb5b\" (UniqueName: \"kubernetes.io/projected/98c3d99d-6fa8-43b3-b76d-3da0abac8968-kube-api-access-ffb5b\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.261816 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.261677    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-home\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.261816 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.261734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tls-certs\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.262134 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.262111    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-model-cache\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.262194 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.262134    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-home\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.262194 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.262158    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tmp-dir\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.264035 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.264000    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-dshm\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.264305 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.264285    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tls-certs\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.268820 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.268796    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffb5b\" (UniqueName: \"kubernetes.io/projected/98c3d99d-6fa8-43b3-b76d-3da0abac8968-kube-api-access-ffb5b\") pod \"sub-test-model-02789d-kserve-66cbb887-df4hx\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") " pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.375113 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.375020    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:46.508252 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:46.508222    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"]
Jul 20 16:33:46.511550 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:33:46.511513    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98c3d99d_6fa8_43b3_b76d_3da0abac8968.slice/crio-0e651c65bfe506736d225fce2d78b854d378dfc4fc75db38505434d8c19a85fa WatchSource:0}: Error finding container 0e651c65bfe506736d225fce2d78b854d378dfc4fc75db38505434d8c19a85fa: Status 404 returned error can't find the container with id 0e651c65bfe506736d225fce2d78b854d378dfc4fc75db38505434d8c19a85fa
Jul 20 16:33:47.168701 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:47.168657    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx" event={"ID":"98c3d99d-6fa8-43b3-b76d-3da0abac8968","Type":"ContainerStarted","Data":"bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55"}
Jul 20 16:33:47.169113 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:47.168727    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx" event={"ID":"98c3d99d-6fa8-43b3-b76d-3da0abac8968","Type":"ContainerStarted","Data":"0e651c65bfe506736d225fce2d78b854d378dfc4fc75db38505434d8c19a85fa"}
Jul 20 16:33:47.188345 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:47.188288    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx" podStartSLOduration=1.188266587 podStartE2EDuration="1.188266587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:33:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:33:47.187086284 +0000 UTC m=+2591.842349559" watchObservedRunningTime="2026-07-20 16:33:47.188266587 +0000 UTC m=+2591.843529865"
Jul 20 16:33:56.375704 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:56.375646    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:56.376169 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:56.375945    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:56.388509 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:56.388484    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:33:57.226324 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:33:57.226293    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:34:05.144980 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:05.144891    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:05.246840 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:05.246804    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:35.797736 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:35.797682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:36.158516 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:36.158429    2564 scope.go:117] "RemoveContainer" containerID="9dd7affb08b1e554dee17b4ebf9c561e90428b9c645361bfd1ed0694c75e3867"
Jul 20 16:34:36.571741 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:36.571679    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"]
Jul 20 16:34:36.579609 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:36.579578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:41.851481 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:41.851446    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:43.377008 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:43.376970    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:43.477541 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:43.477507    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:47.787297 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:47.787259    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:49.650166 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:49.650127    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:49.750334 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:49.750302    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:51.592216 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.592173    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx" podUID="98c3d99d-6fa8-43b3-b76d-3da0abac8968" containerName="main" containerID="cri-o://bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55" gracePeriod=45
Jul 20 16:34:51.848206 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.848143    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:34:51.971033 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.970994    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tmp-dir\") pod \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") "
Jul 20 16:34:51.971236 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.971069    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffb5b\" (UniqueName: \"kubernetes.io/projected/98c3d99d-6fa8-43b3-b76d-3da0abac8968-kube-api-access-ffb5b\") pod \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") "
Jul 20 16:34:51.971236 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.971104    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tls-certs\") pod \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") "
Jul 20 16:34:51.971236 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.971132    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-model-cache\") pod \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") "
Jul 20 16:34:51.971236 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.971152    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-home\") pod \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") "
Jul 20 16:34:51.971236 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.971183    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-dshm\") pod \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\" (UID: \"98c3d99d-6fa8-43b3-b76d-3da0abac8968\") "
Jul 20 16:34:51.971510 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.971327    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98c3d99d-6fa8-43b3-b76d-3da0abac8968" (UID: "98c3d99d-6fa8-43b3-b76d-3da0abac8968"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:34:51.971510 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.971424    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-model-cache" (OuterVolumeSpecName: "model-cache") pod "98c3d99d-6fa8-43b3-b76d-3da0abac8968" (UID: "98c3d99d-6fa8-43b3-b76d-3da0abac8968"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:34:51.971510 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.971446    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-home" (OuterVolumeSpecName: "home") pod "98c3d99d-6fa8-43b3-b76d-3da0abac8968" (UID: "98c3d99d-6fa8-43b3-b76d-3da0abac8968"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:34:51.971621 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.971500    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tmp-dir\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:34:51.973366 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.973337    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98c3d99d-6fa8-43b3-b76d-3da0abac8968" (UID: "98c3d99d-6fa8-43b3-b76d-3da0abac8968"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:34:51.973502 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.973397    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-dshm" (OuterVolumeSpecName: "dshm") pod "98c3d99d-6fa8-43b3-b76d-3da0abac8968" (UID: "98c3d99d-6fa8-43b3-b76d-3da0abac8968"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:34:51.973502 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:51.973457    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c3d99d-6fa8-43b3-b76d-3da0abac8968-kube-api-access-ffb5b" (OuterVolumeSpecName: "kube-api-access-ffb5b") pod "98c3d99d-6fa8-43b3-b76d-3da0abac8968" (UID: "98c3d99d-6fa8-43b3-b76d-3da0abac8968"). InnerVolumeSpecName "kube-api-access-ffb5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:34:52.072597 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.072557    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-dshm\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:34:52.072597 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.072593    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffb5b\" (UniqueName: \"kubernetes.io/projected/98c3d99d-6fa8-43b3-b76d-3da0abac8968-kube-api-access-ffb5b\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:34:52.072597 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.072604    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c3d99d-6fa8-43b3-b76d-3da0abac8968-tls-certs\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:34:52.072879 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.072615    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-model-cache\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:34:52.072879 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.072623    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98c3d99d-6fa8-43b3-b76d-3da0abac8968-home\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:34:52.473221 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.473170    2564 generic.go:356] "Generic (PLEG): container finished" podID="98c3d99d-6fa8-43b3-b76d-3da0abac8968" containerID="bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55" exitCode=0
Jul 20 16:34:52.473403 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.473248    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx" event={"ID":"98c3d99d-6fa8-43b3-b76d-3da0abac8968","Type":"ContainerDied","Data":"bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55"}
Jul 20 16:34:52.473403 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.473287    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx" event={"ID":"98c3d99d-6fa8-43b3-b76d-3da0abac8968","Type":"ContainerDied","Data":"0e651c65bfe506736d225fce2d78b854d378dfc4fc75db38505434d8c19a85fa"}
Jul 20 16:34:52.473403 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.473300    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"
Jul 20 16:34:52.473403 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.473314    2564 scope.go:117] "RemoveContainer" containerID="bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55"
Jul 20 16:34:52.483785 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.483766    2564 scope.go:117] "RemoveContainer" containerID="bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55"
Jul 20 16:34:52.484118 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:34:52.484094    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55\": container with ID starting with bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55 not found: ID does not exist" containerID="bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55"
Jul 20 16:34:52.484172 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.484130    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55"} err="failed to get container status \"bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55\": rpc error: code = NotFound desc = could not find container \"bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55\": container with ID starting with bd21194c482c146f35cdc6fe1455c1e1cda56fa5e83d4b6ca1f7ec85a877ef55 not found: ID does not exist"
Jul 20 16:34:52.492640 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.492587    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"]
Jul 20 16:34:52.494624 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:52.494599    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/sub-test-model-02789d-kserve-66cbb887-df4hx"]
Jul 20 16:34:53.869313 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:34:53.869278    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98c3d99d-6fa8-43b3-b76d-3da0abac8968" path="/var/lib/kubelet/pods/98c3d99d-6fa8-43b3-b76d-3da0abac8968/volumes"
Jul 20 16:35:09.371267 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.371232    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"]
Jul 20 16:35:09.371781 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.371643    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c3d99d-6fa8-43b3-b76d-3da0abac8968" containerName="main"
Jul 20 16:35:09.371781 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.371658    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c3d99d-6fa8-43b3-b76d-3da0abac8968" containerName="main"
Jul 20 16:35:09.371781 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.371751    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c3d99d-6fa8-43b3-b76d-3da0abac8968" containerName="main"
Jul 20 16:35:09.376147 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.376122    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.378234 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.378212    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"openshift-service-ca.crt\""
Jul 20 16:35:09.378343 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.378242    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"default-dockercfg-b7wm4\""
Jul 20 16:35:09.378906 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.378890    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"kube-root-ca.crt\""
Jul 20 16:35:09.378993 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.378976    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-02789d\"/\"rate-test-model-02789d-kserve-self-signed-certs\""
Jul 20 16:35:09.385393 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.385368    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"]
Jul 20 16:35:09.407806 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.407768    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:09.525150 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.525103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tmp-dir\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.525150 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.525155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-model-cache\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.525399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.525190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-dshm\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.525399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.525217    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r55q\" (UniqueName: \"kubernetes.io/projected/60b105fa-4635-4f7a-aeb2-d0420a885dfa-kube-api-access-9r55q\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.525399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.525245    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-home\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.525399 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.525266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tls-certs\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.626650 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.626562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tmp-dir\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.626650 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.626603    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-model-cache\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.626650 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.626632    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-dshm\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.626967 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.626764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r55q\" (UniqueName: \"kubernetes.io/projected/60b105fa-4635-4f7a-aeb2-d0420a885dfa-kube-api-access-9r55q\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.626967 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.626813    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-home\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.627084 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.626967    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tls-certs\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.627151 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.627108    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-model-cache\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.627151 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.627137    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tmp-dir\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.627222 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.627156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-home\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.628903 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.628885    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-dshm\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.629347 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.629329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tls-certs\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.636414 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.636386    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r55q\" (UniqueName: \"kubernetes.io/projected/60b105fa-4635-4f7a-aeb2-d0420a885dfa-kube-api-access-9r55q\") pod \"rate-test-model-02789d-kserve-5b9fb64f99-lncll\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") " pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.685449 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.685405    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:09.824156 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:09.824126    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"]
Jul 20 16:35:09.826944 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:35:09.826920    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60b105fa_4635_4f7a_aeb2_d0420a885dfa.slice/crio-d626d1c9d61f03d2e278df7c998eb95b77184e1f49eed632da376f49bcaad078 WatchSource:0}: Error finding container d626d1c9d61f03d2e278df7c998eb95b77184e1f49eed632da376f49bcaad078: Status 404 returned error can't find the container with id d626d1c9d61f03d2e278df7c998eb95b77184e1f49eed632da376f49bcaad078
Jul 20 16:35:10.557402 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:10.557366    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll" event={"ID":"60b105fa-4635-4f7a-aeb2-d0420a885dfa","Type":"ContainerStarted","Data":"8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859"}
Jul 20 16:35:10.557402 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:10.557409    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll" event={"ID":"60b105fa-4635-4f7a-aeb2-d0420a885dfa","Type":"ContainerStarted","Data":"d626d1c9d61f03d2e278df7c998eb95b77184e1f49eed632da376f49bcaad078"}
Jul 20 16:35:10.574400 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:10.574350    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll" podStartSLOduration=1.574333701 podStartE2EDuration="1.574333701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:35:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:35:10.572751505 +0000 UTC m=+2675.228014780" watchObservedRunningTime="2026-07-20 16:35:10.574333701 +0000 UTC m=+2675.229596976"
Jul 20 16:35:19.685881 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:19.685837    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:19.685881 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:19.685890    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:19.698283 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:19.698255    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:20.615067 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:20.615034    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:35:35.967946 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:35.967864    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:35:35.969849 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:35.969822    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxp2n_6310153e-76c1-4dc3-bfc2-e36cc217b2e1/ovn-acl-logging/0.log"
Jul 20 16:35:36.178152 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:36.178118    2564 scope.go:117] "RemoveContainer" containerID="22c76cccb4e14784ae78e9053079e8312170a07128fca2e5c1543a420f2cf4be"
Jul 20 16:35:36.186297 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:36.186277    2564 scope.go:117] "RemoveContainer" containerID="84521e334ac86e5b934a04266e930a32f1571654f0a74cc84a69212f6d0c48bd"
Jul 20 16:35:48.788241 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:48.788200    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:48.839466 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:48.839437    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:53.581506 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:53.581457    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:54.372524 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:54.372485    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"]
Jul 20 16:35:54.394545 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:54.394511    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:56.843138 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:56.843095    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:56.944345 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:35:56.944307    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:09.393977 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.393923    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll" podUID="60b105fa-4635-4f7a-aeb2-d0420a885dfa" containerName="main" containerID="cri-o://8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859" gracePeriod=45
Jul 20 16:36:09.647435 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.647356    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:36:09.743304 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.743264    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-model-cache\") pod \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") "
Jul 20 16:36:09.743491 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.743318    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-dshm\") pod \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") "
Jul 20 16:36:09.743491 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.743360    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r55q\" (UniqueName: \"kubernetes.io/projected/60b105fa-4635-4f7a-aeb2-d0420a885dfa-kube-api-access-9r55q\") pod \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") "
Jul 20 16:36:09.743491 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.743399    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tmp-dir\") pod \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") "
Jul 20 16:36:09.743491 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.743418    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tls-certs\") pod \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") "
Jul 20 16:36:09.743491 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.743463    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-home\") pod \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\" (UID: \"60b105fa-4635-4f7a-aeb2-d0420a885dfa\") "
Jul 20 16:36:09.743717 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.743598    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-model-cache" (OuterVolumeSpecName: "model-cache") pod "60b105fa-4635-4f7a-aeb2-d0420a885dfa" (UID: "60b105fa-4635-4f7a-aeb2-d0420a885dfa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:36:09.743717 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.743635    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60b105fa-4635-4f7a-aeb2-d0420a885dfa" (UID: "60b105fa-4635-4f7a-aeb2-d0420a885dfa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:36:09.743885 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.743862    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-home" (OuterVolumeSpecName: "home") pod "60b105fa-4635-4f7a-aeb2-d0420a885dfa" (UID: "60b105fa-4635-4f7a-aeb2-d0420a885dfa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:36:09.745542 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.745514    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60b105fa-4635-4f7a-aeb2-d0420a885dfa-kube-api-access-9r55q" (OuterVolumeSpecName: "kube-api-access-9r55q") pod "60b105fa-4635-4f7a-aeb2-d0420a885dfa" (UID: "60b105fa-4635-4f7a-aeb2-d0420a885dfa"). InnerVolumeSpecName "kube-api-access-9r55q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:36:09.745542 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.745529    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60b105fa-4635-4f7a-aeb2-d0420a885dfa" (UID: "60b105fa-4635-4f7a-aeb2-d0420a885dfa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:36:09.745718 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.745518    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-dshm" (OuterVolumeSpecName: "dshm") pod "60b105fa-4635-4f7a-aeb2-d0420a885dfa" (UID: "60b105fa-4635-4f7a-aeb2-d0420a885dfa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:36:09.829291 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.829256    2564 generic.go:356] "Generic (PLEG): container finished" podID="60b105fa-4635-4f7a-aeb2-d0420a885dfa" containerID="8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859" exitCode=0
Jul 20 16:36:09.829477 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.829300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll" event={"ID":"60b105fa-4635-4f7a-aeb2-d0420a885dfa","Type":"ContainerDied","Data":"8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859"}
Jul 20 16:36:09.829477 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.829305    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"
Jul 20 16:36:09.829477 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.829331    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll" event={"ID":"60b105fa-4635-4f7a-aeb2-d0420a885dfa","Type":"ContainerDied","Data":"d626d1c9d61f03d2e278df7c998eb95b77184e1f49eed632da376f49bcaad078"}
Jul 20 16:36:09.829477 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.829355    2564 scope.go:117] "RemoveContainer" containerID="8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859"
Jul 20 16:36:09.839013 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.838992    2564 scope.go:117] "RemoveContainer" containerID="8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859"
Jul 20 16:36:09.839307 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:36:09.839288    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859\": container with ID starting with 8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859 not found: ID does not exist" containerID="8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859"
Jul 20 16:36:09.839384 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.839330    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859"} err="failed to get container status \"8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859\": rpc error: code = NotFound desc = could not find container \"8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859\": container with ID starting with 8a996bb72ad5ee5379fc19f8991e0b5afd6b0368b8c3537fb95f92a945bf0859 not found: ID does not exist"
Jul 20 16:36:09.844225 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.844202    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-home\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:09.844312 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.844226    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-model-cache\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:09.844312 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.844236    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-dshm\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:09.844312 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.844246    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r55q\" (UniqueName: \"kubernetes.io/projected/60b105fa-4635-4f7a-aeb2-d0420a885dfa-kube-api-access-9r55q\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:09.844312 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.844254    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tmp-dir\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:09.844312 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.844262    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b105fa-4635-4f7a-aeb2-d0420a885dfa-tls-certs\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:09.849919 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.849894    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"]
Jul 20 16:36:09.857481 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.857453    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-02789d/rate-test-model-02789d-kserve-5b9fb64f99-lncll"]
Jul 20 16:36:09.869640 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:09.869605    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60b105fa-4635-4f7a-aeb2-d0420a885dfa" path="/var/lib/kubelet/pods/60b105fa-4635-4f7a-aeb2-d0420a885dfa/volumes"
Jul 20 16:36:13.087012 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.086975    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"]
Jul 20 16:36:13.087557 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.087318    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60b105fa-4635-4f7a-aeb2-d0420a885dfa" containerName="main"
Jul 20 16:36:13.087557 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.087329    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="60b105fa-4635-4f7a-aeb2-d0420a885dfa" containerName="main"
Jul 20 16:36:13.087557 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.087391    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="60b105fa-4635-4f7a-aeb2-d0420a885dfa" containerName="main"
Jul 20 16:36:13.091420 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.091399    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"
Jul 20 16:36:13.093517 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.093496    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:36:13.098906 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.098874    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"]
Jul 20 16:36:13.275945 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.275905    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28a1c48-18e5-49ae-9813-a6c8e9360f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7\" (UID: \"a28a1c48-18e5-49ae-9813-a6c8e9360f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"
Jul 20 16:36:13.376829 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.376740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28a1c48-18e5-49ae-9813-a6c8e9360f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7\" (UID: \"a28a1c48-18e5-49ae-9813-a6c8e9360f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"
Jul 20 16:36:13.377354 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.377333    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28a1c48-18e5-49ae-9813-a6c8e9360f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7\" (UID: \"a28a1c48-18e5-49ae-9813-a6c8e9360f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"
Jul 20 16:36:13.404873 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.404840    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"
Jul 20 16:36:13.545227 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.545201    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"]
Jul 20 16:36:13.547832 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:36:13.547805    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28a1c48_18e5_49ae_9813_a6c8e9360f9f.slice/crio-6a70768b9af5f4063ee7e09e9b43c911545eb3ba5743f6ba211735f9100c1986 WatchSource:0}: Error finding container 6a70768b9af5f4063ee7e09e9b43c911545eb3ba5743f6ba211735f9100c1986: Status 404 returned error can't find the container with id 6a70768b9af5f4063ee7e09e9b43c911545eb3ba5743f6ba211735f9100c1986
Jul 20 16:36:13.595001 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.594945    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:13.688622 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.688523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:13.849392 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.849351    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7" event={"ID":"a28a1c48-18e5-49ae-9813-a6c8e9360f9f","Type":"ContainerStarted","Data":"7acfe6f3015fbc44addc94f2e8a08cae9b3ed245e83663bbf264d727676a0f34"}
Jul 20 16:36:13.849571 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.849414    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7" event={"ID":"a28a1c48-18e5-49ae-9813-a6c8e9360f9f","Type":"ContainerStarted","Data":"6a70768b9af5f4063ee7e09e9b43c911545eb3ba5743f6ba211735f9100c1986"}
Jul 20 16:36:13.865522 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:13.865468    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7" podStartSLOduration=0.865450657 podStartE2EDuration="865.450657ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:36:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:36:13.863735233 +0000 UTC m=+2738.518998507" watchObservedRunningTime="2026-07-20 16:36:13.865450657 +0000 UTC m=+2738.520713932"
Jul 20 16:36:14.855765 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:14.855727    2564 generic.go:356] "Generic (PLEG): container finished" podID="a28a1c48-18e5-49ae-9813-a6c8e9360f9f" containerID="7acfe6f3015fbc44addc94f2e8a08cae9b3ed245e83663bbf264d727676a0f34" exitCode=0
Jul 20 16:36:14.856280 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:14.855828    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7" event={"ID":"a28a1c48-18e5-49ae-9813-a6c8e9360f9f","Type":"ContainerDied","Data":"7acfe6f3015fbc44addc94f2e8a08cae9b3ed245e83663bbf264d727676a0f34"}
Jul 20 16:36:15.996030 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:15.996002    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"
Jul 20 16:36:16.100072 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:16.100036    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28a1c48-18e5-49ae-9813-a6c8e9360f9f-maas-api-service-ca\") pod \"a28a1c48-18e5-49ae-9813-a6c8e9360f9f\" (UID: \"a28a1c48-18e5-49ae-9813-a6c8e9360f9f\") "
Jul 20 16:36:16.100433 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:16.100411    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a28a1c48-18e5-49ae-9813-a6c8e9360f9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a28a1c48-18e5-49ae-9813-a6c8e9360f9f" (UID: "a28a1c48-18e5-49ae-9813-a6c8e9360f9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:36:16.201588 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:16.201496    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28a1c48-18e5-49ae-9813-a6c8e9360f9f-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:16.872521 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:16.872488    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"
Jul 20 16:36:16.872715 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:16.872487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7" event={"ID":"a28a1c48-18e5-49ae-9813-a6c8e9360f9f","Type":"ContainerDied","Data":"6a70768b9af5f4063ee7e09e9b43c911545eb3ba5743f6ba211735f9100c1986"}
Jul 20 16:36:16.872715 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:16.872639    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a70768b9af5f4063ee7e09e9b43c911545eb3ba5743f6ba211735f9100c1986"
Jul 20 16:36:23.566028 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.565989    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"]
Jul 20 16:36:23.566915 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.566860    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs" podUID="5f63d717-1ca0-4ba9-ad7a-a059b178f164" containerName="maas-api" containerID="cri-o://7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28" gracePeriod=30
Jul 20 16:36:23.586171 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.586138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:23.822233 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.822164    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:36:23.852975 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.852939    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7ddg\" (UniqueName: \"kubernetes.io/projected/5f63d717-1ca0-4ba9-ad7a-a059b178f164-kube-api-access-z7ddg\") pod \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") "
Jul 20 16:36:23.853168 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.852999    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f63d717-1ca0-4ba9-ad7a-a059b178f164-openshift-service-ca\") pod \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") "
Jul 20 16:36:23.853364 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.853334    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f63d717-1ca0-4ba9-ad7a-a059b178f164-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5f63d717-1ca0-4ba9-ad7a-a059b178f164" (UID: "5f63d717-1ca0-4ba9-ad7a-a059b178f164"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:36:23.855200 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.855167    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f63d717-1ca0-4ba9-ad7a-a059b178f164-kube-api-access-z7ddg" (OuterVolumeSpecName: "kube-api-access-z7ddg") pod "5f63d717-1ca0-4ba9-ad7a-a059b178f164" (UID: "5f63d717-1ca0-4ba9-ad7a-a059b178f164"). InnerVolumeSpecName "kube-api-access-z7ddg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:36:23.907167 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.907122    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f63d717-1ca0-4ba9-ad7a-a059b178f164" containerID="7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28" exitCode=0
Jul 20 16:36:23.907333 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.907172    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"
Jul 20 16:36:23.907333 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.907208    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs" event={"ID":"5f63d717-1ca0-4ba9-ad7a-a059b178f164","Type":"ContainerDied","Data":"7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28"}
Jul 20 16:36:23.907333 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.907264    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs" event={"ID":"5f63d717-1ca0-4ba9-ad7a-a059b178f164","Type":"ContainerDied","Data":"68787857662b2e2b2ae5efe3d8966d4b0f641006c2aa2b35f159460401c3bcb8"}
Jul 20 16:36:23.907333 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.907292    2564 scope.go:117] "RemoveContainer" containerID="7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28"
Jul 20 16:36:23.916480 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.916461    2564 scope.go:117] "RemoveContainer" containerID="7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28"
Jul 20 16:36:23.916788 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:36:23.916765    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28\": container with ID starting with 7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28 not found: ID does not exist" containerID="7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28"
Jul 20 16:36:23.916883 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.916797    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28"} err="failed to get container status \"7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28\": rpc error: code = NotFound desc = could not find container \"7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28\": container with ID starting with 7152099c505fe2eead4e9aa070bcb3e33d8e4208dd397d40db21628f8de8cf28 not found: ID does not exist"
Jul 20 16:36:23.953805 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.953779    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f63d717-1ca0-4ba9-ad7a-a059b178f164-maas-api-tls\") pod \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\" (UID: \"5f63d717-1ca0-4ba9-ad7a-a059b178f164\") "
Jul 20 16:36:23.954126 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.954106    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7ddg\" (UniqueName: \"kubernetes.io/projected/5f63d717-1ca0-4ba9-ad7a-a059b178f164-kube-api-access-z7ddg\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:23.954166 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.954133    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f63d717-1ca0-4ba9-ad7a-a059b178f164-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:23.955811 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:23.955788    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f63d717-1ca0-4ba9-ad7a-a059b178f164-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5f63d717-1ca0-4ba9-ad7a-a059b178f164" (UID: "5f63d717-1ca0-4ba9-ad7a-a059b178f164"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:36:24.055300 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:24.055263    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f63d717-1ca0-4ba9-ad7a-a059b178f164-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:24.231247 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:24.231214    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"]
Jul 20 16:36:24.232814 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:24.232793    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-329d3d-7874fbb89d-95phs"]
Jul 20 16:36:25.869086 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:25.869050    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f63d717-1ca0-4ba9-ad7a-a059b178f164" path="/var/lib/kubelet/pods/5f63d717-1ca0-4ba9-ad7a-a059b178f164/volumes"
Jul 20 16:36:28.165375 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:28.165335    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:28.177841 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:28.177810    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"]
Jul 20 16:36:28.180424 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:28.180399    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-329d3d-ldzh7"]
Jul 20 16:36:28.560844 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:28.560812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:29.868550 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:29.868516    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a28a1c48-18e5-49ae-9813-a6c8e9360f9f" path="/var/lib/kubelet/pods/a28a1c48-18e5-49ae-9813-a6c8e9360f9f/volumes"
Jul 20 16:36:34.449888 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:34.449845    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:34.500818 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:34.500778    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:34.708244 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:34.708151    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:34.921701 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:34.921664    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:35.046673 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.046636    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"]
Jul 20 16:36:35.047002 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.046989    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f63d717-1ca0-4ba9-ad7a-a059b178f164" containerName="maas-api"
Jul 20 16:36:35.047048 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.047003    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f63d717-1ca0-4ba9-ad7a-a059b178f164" containerName="maas-api"
Jul 20 16:36:35.047048 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.047019    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a28a1c48-18e5-49ae-9813-a6c8e9360f9f" containerName="revoke-keys"
Jul 20 16:36:35.047048 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.047025    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28a1c48-18e5-49ae-9813-a6c8e9360f9f" containerName="revoke-keys"
Jul 20 16:36:35.047164 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.047079    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a28a1c48-18e5-49ae-9813-a6c8e9360f9f" containerName="revoke-keys"
Jul 20 16:36:35.047164 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.047089    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f63d717-1ca0-4ba9-ad7a-a059b178f164" containerName="maas-api"
Jul 20 16:36:35.051359 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.051338    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"
Jul 20 16:36:35.053581 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.053556    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\""
Jul 20 16:36:35.056609 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.056583    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"]
Jul 20 16:36:35.141568 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.141530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46dc6874-31b8-4cd2-b770-bb0d3a671c8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-02789d-9fggt\" (UID: \"46dc6874-31b8-4cd2-b770-bb0d3a671c8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"
Jul 20 16:36:35.242865 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.242823    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46dc6874-31b8-4cd2-b770-bb0d3a671c8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-02789d-9fggt\" (UID: \"46dc6874-31b8-4cd2-b770-bb0d3a671c8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"
Jul 20 16:36:35.243469 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.243447    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46dc6874-31b8-4cd2-b770-bb0d3a671c8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-02789d-9fggt\" (UID: \"46dc6874-31b8-4cd2-b770-bb0d3a671c8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"
Jul 20 16:36:35.363390 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.363299    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"
Jul 20 16:36:35.512094 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.512066    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"]
Jul 20 16:36:35.514487 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:36:35.514456    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46dc6874_31b8_4cd2_b770_bb0d3a671c8e.slice/crio-b74d783a56ff5117d4acc27a798f9ad3d9702cd383b13ffa66976e74d284e570 WatchSource:0}: Error finding container b74d783a56ff5117d4acc27a798f9ad3d9702cd383b13ffa66976e74d284e570: Status 404 returned error can't find the container with id b74d783a56ff5117d4acc27a798f9ad3d9702cd383b13ffa66976e74d284e570
Jul 20 16:36:35.817093 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.817053    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:35.918344 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.918304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:35.966774 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.966738    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt" event={"ID":"46dc6874-31b8-4cd2-b770-bb0d3a671c8e","Type":"ContainerStarted","Data":"0fb38253afdf9e9081b63a0382eda3fb789e443ea23f7d7a09501f1a14736b62"}
Jul 20 16:36:35.966952 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.966788    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt" event={"ID":"46dc6874-31b8-4cd2-b770-bb0d3a671c8e","Type":"ContainerStarted","Data":"b74d783a56ff5117d4acc27a798f9ad3d9702cd383b13ffa66976e74d284e570"}
Jul 20 16:36:35.981794 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:35.981670    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt" podStartSLOduration=0.981651485 podStartE2EDuration="981.651485ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:36:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:36:35.980849287 +0000 UTC m=+2760.636112562" watchObservedRunningTime="2026-07-20 16:36:35.981651485 +0000 UTC m=+2760.636914760"
Jul 20 16:36:36.211119 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:36.211080    2564 scope.go:117] "RemoveContainer" containerID="9fc370a7b6c05b27f9f78409c186d6c8ae4f87adbcbf0d2e3be94495af5d609c"
Jul 20 16:36:36.262010 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:36.261985    2564 scope.go:117] "RemoveContainer" containerID="c0b14662875b9eea6a2aae7b08bdac2e1f2dc0398f6e3ddf6226e8855973042f"
Jul 20 16:36:36.973030 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:36.972994    2564 generic.go:356] "Generic (PLEG): container finished" podID="46dc6874-31b8-4cd2-b770-bb0d3a671c8e" containerID="0fb38253afdf9e9081b63a0382eda3fb789e443ea23f7d7a09501f1a14736b62" exitCode=0
Jul 20 16:36:36.973433 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:36.973079    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt" event={"ID":"46dc6874-31b8-4cd2-b770-bb0d3a671c8e","Type":"ContainerDied","Data":"0fb38253afdf9e9081b63a0382eda3fb789e443ea23f7d7a09501f1a14736b62"}
Jul 20 16:36:38.117368 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:38.117338    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"
Jul 20 16:36:38.169059 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:38.169022    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46dc6874-31b8-4cd2-b770-bb0d3a671c8e-maas-api-service-ca\") pod \"46dc6874-31b8-4cd2-b770-bb0d3a671c8e\" (UID: \"46dc6874-31b8-4cd2-b770-bb0d3a671c8e\") "
Jul 20 16:36:38.169385 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:38.169359    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46dc6874-31b8-4cd2-b770-bb0d3a671c8e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "46dc6874-31b8-4cd2-b770-bb0d3a671c8e" (UID: "46dc6874-31b8-4cd2-b770-bb0d3a671c8e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:36:38.270057 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:38.270025    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46dc6874-31b8-4cd2-b770-bb0d3a671c8e-maas-api-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:38.983188 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:38.983160    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"
Jul 20 16:36:38.983384 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:38.983196    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt" event={"ID":"46dc6874-31b8-4cd2-b770-bb0d3a671c8e","Type":"ContainerDied","Data":"b74d783a56ff5117d4acc27a798f9ad3d9702cd383b13ffa66976e74d284e570"}
Jul 20 16:36:38.983384 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:38.983252    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b74d783a56ff5117d4acc27a798f9ad3d9702cd383b13ffa66976e74d284e570"
Jul 20 16:36:45.580743 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:45.580676    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"]
Jul 20 16:36:45.581658 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:45.581601    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4" podUID="9b71a6a3-e0bd-47fd-86fe-02841fe52cbc" containerName="maas-api" containerID="cri-o://68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e" gracePeriod=30
Jul 20 16:36:45.608557 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:45.608523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:45.833114 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:45.833047    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:36:45.935081 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:45.935045    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-openshift-service-ca\") pod \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") "
Jul 20 16:36:45.935291 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:45.935115    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-maas-api-tls\") pod \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") "
Jul 20 16:36:45.935291 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:45.935148    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6fj9\" (UniqueName: \"kubernetes.io/projected/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-kube-api-access-j6fj9\") pod \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\" (UID: \"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc\") "
Jul 20 16:36:45.935497 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:45.935468    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b71a6a3-e0bd-47fd-86fe-02841fe52cbc" (UID: "9b71a6a3-e0bd-47fd-86fe-02841fe52cbc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:36:45.937341 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:45.937315    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b71a6a3-e0bd-47fd-86fe-02841fe52cbc" (UID: "9b71a6a3-e0bd-47fd-86fe-02841fe52cbc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:36:45.937544 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:45.937523    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-kube-api-access-j6fj9" (OuterVolumeSpecName: "kube-api-access-j6fj9") pod "9b71a6a3-e0bd-47fd-86fe-02841fe52cbc" (UID: "9b71a6a3-e0bd-47fd-86fe-02841fe52cbc"). InnerVolumeSpecName "kube-api-access-j6fj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:36:46.014324 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.014289    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b71a6a3-e0bd-47fd-86fe-02841fe52cbc" containerID="68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e" exitCode=0
Jul 20 16:36:46.014493 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.014336    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4" event={"ID":"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc","Type":"ContainerDied","Data":"68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e"}
Jul 20 16:36:46.014493 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.014344    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"
Jul 20 16:36:46.014493 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.014385    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4" event={"ID":"9b71a6a3-e0bd-47fd-86fe-02841fe52cbc","Type":"ContainerDied","Data":"e14ed0b517bba65fe41cfb6ccbcc77aa0537e0b7afca8317bcb1177a4b572407"}
Jul 20 16:36:46.014493 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.014408    2564 scope.go:117] "RemoveContainer" containerID="68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e"
Jul 20 16:36:46.023179 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.023158    2564 scope.go:117] "RemoveContainer" containerID="68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e"
Jul 20 16:36:46.023464 ip-10-0-134-220 kubenswrapper[2564]: E0720 16:36:46.023445    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e\": container with ID starting with 68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e not found: ID does not exist" containerID="68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e"
Jul 20 16:36:46.023503 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.023476    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e"} err="failed to get container status \"68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e\": rpc error: code = NotFound desc = could not find container \"68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e\": container with ID starting with 68b90a4a857379832d56dc832b5bdb5a664d45052b310aa9b4cbbe07b8241a6e not found: ID does not exist"
Jul 20 16:36:46.036629 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.036588    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:46.036629 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.036628    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:46.036956 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.036651    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6fj9\" (UniqueName: \"kubernetes.io/projected/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc-kube-api-access-j6fj9\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\""
Jul 20 16:36:46.038730 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.038703    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"]
Jul 20 16:36:46.045380 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:46.045351    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-02789d-68d94ccfdb-h8kd4"]
Jul 20 16:36:47.868845 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:47.868812    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b71a6a3-e0bd-47fd-86fe-02841fe52cbc" path="/var/lib/kubelet/pods/9b71a6a3-e0bd-47fd-86fe-02841fe52cbc/volumes"
Jul 20 16:36:50.140465 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:50.140428    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"]
Jul 20 16:36:50.141598 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:50.141570    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:50.143566 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:50.143544    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-02789d-9fggt"]
Jul 20 16:36:50.507882 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:50.507788    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:51.868755 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:51.868713    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46dc6874-31b8-4cd2-b770-bb0d3a671c8e" path="/var/lib/kubelet/pods/46dc6874-31b8-4cd2-b770-bb0d3a671c8e/volumes"
Jul 20 16:36:56.818845 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:56.818802    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:56.913929 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:56.913889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:57.122115 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:57.122006    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:57.341412 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:36:57.341374    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:03.149600 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:03.149524    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9xmww_589ce640-c965-4058-87c3-e2e3eb813352/manager/0.log"
Jul 20 16:37:03.269945 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:03.269914    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pr94l_523b88b3-dd29-495c-b568-d04767ffe45a/manager/0.log"
Jul 20 16:37:03.403304 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:03.403203    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76d8696695-zq55p_ba52a7b9-3b00-4304-9891-a43d97f47cf4/manager/0.log"
Jul 20 16:37:03.753097 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:03.753013    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b76897bc6-f8cng_623fe211-2e84-4558-bbda-856abfa7a705/manager/0.log"
Jul 20 16:37:03.763581 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:03.763551    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b76897bc6-f8cng_623fe211-2e84-4558-bbda-856abfa7a705/copy-manifests/0.log"
Jul 20 16:37:04.340478 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:04.340446    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-fbf94c8c9-wtqvz_ce30f998-a0d9-431f-af09-2c5a7ac8b52d/maas-api/0.log"
Jul 20 16:37:04.552529 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:04.552499    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742735-wgtgw_6fd531f4-33fd-4fcf-8468-640d2bb56578/cleanup/0.log"
Jul 20 16:37:04.664309 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:04.664219    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742750-tlbmq_4ce871b0-0fec-42a9-a45b-eae92687c2c8/cleanup/0.log"
Jul 20 16:37:04.776228 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:04.776197    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-xh2wg_035ec57e-3ecf-4030-81d1-72d263f46989/postgres/0.log"
Jul 20 16:37:06.903697 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:06.903658    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jspg4_fd3b5025-3574-410f-8634-e0858fc5e985/manager/0.log"
Jul 20 16:37:07.380271 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:07.380215    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hpbz4_1a0c8fb0-d84c-432c-96f3-55f0e5312188/discovery/0.log"
Jul 20 16:37:07.610223 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:07.610192    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bc6c54cc6-rb9cf_8f0545fb-6ba1-45c6-8fac-d6717a671111/kube-auth-proxy/0.log"
Jul 20 16:37:07.984749 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:07.984717    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lfsj2_805b55dd-0b56-4831-8165-29ef025d779a/payload-processing/0.log"
Jul 20 16:37:08.559404 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:08.559367    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-b8ggb_0280a46a-5bea-4910-a0e0-bcc00b2380b4/main/0.log"
Jul 20 16:37:08.775141 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:08.775102    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2nrbm_ab0f2ac0-2343-49ff-9d28-bcd8f0e38a10/main/0.log"
Jul 20 16:37:14.652056 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:14.652013    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:14.751576 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:14.751539    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:18.250864 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:18.250832    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t4wh7_672c6793-5272-43d1-aa5d-ce5bf204f783/global-pull-secret-syncer/0.log"
Jul 20 16:37:18.321664 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:18.321631    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-99khn_73455279-4b14-4bd5-923a-a23b34bd843c/konnectivity-agent/0.log"
Jul 20 16:37:18.424405 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:18.424374    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-220.ec2.internal_c2bd98933375e305b9c32417ca912b54/haproxy/0.log"
Jul 20 16:37:24.410730 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:24.410679    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jspg4_fd3b5025-3574-410f-8634-e0858fc5e985/manager/0.log"
Jul 20 16:37:26.559975 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:26.559946    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cztdc_2cd283b7-ea37-49fa-9ab6-adbc9909c8f0/node-exporter/0.log"
Jul 20 16:37:26.583676 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:26.583643    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cztdc_2cd283b7-ea37-49fa-9ab6-adbc9909c8f0/kube-rbac-proxy/0.log"
Jul 20 16:37:26.607645 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:26.607620    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cztdc_2cd283b7-ea37-49fa-9ab6-adbc9909c8f0/init-textfile/0.log"
Jul 20 16:37:28.447740 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.447679    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"]
Jul 20 16:37:28.448155 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.448019    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46dc6874-31b8-4cd2-b770-bb0d3a671c8e" containerName="revoke-keys"
Jul 20 16:37:28.448155 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.448030    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="46dc6874-31b8-4cd2-b770-bb0d3a671c8e" containerName="revoke-keys"
Jul 20 16:37:28.448155 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.448041    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b71a6a3-e0bd-47fd-86fe-02841fe52cbc" containerName="maas-api"
Jul 20 16:37:28.448155 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.448047    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b71a6a3-e0bd-47fd-86fe-02841fe52cbc" containerName="maas-api"
Jul 20 16:37:28.448155 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.448110    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b71a6a3-e0bd-47fd-86fe-02841fe52cbc" containerName="maas-api"
Jul 20 16:37:28.448155 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.448116    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="46dc6874-31b8-4cd2-b770-bb0d3a671c8e" containerName="revoke-keys"
Jul 20 16:37:28.451082 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.451064    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.454002 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.453976    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q8qg5\"/\"default-dockercfg-4bfpg\""
Jul 20 16:37:28.454227 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.454208    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q8qg5\"/\"openshift-service-ca.crt\""
Jul 20 16:37:28.454482 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.454469    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q8qg5\"/\"kube-root-ca.crt\""
Jul 20 16:37:28.479739 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.479698    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"]
Jul 20 16:37:28.507220 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.507179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-proc\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.507220 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.507222    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-podres\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.507440 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.507257    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh6gs\" (UniqueName: \"kubernetes.io/projected/2cfddf9c-f96e-498f-8ffc-bec641c097d7-kube-api-access-zh6gs\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.507440 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.507286    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-lib-modules\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.507440 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.507346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-sys\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.608515 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.608467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-podres\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.608515 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.608517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh6gs\" (UniqueName: \"kubernetes.io/projected/2cfddf9c-f96e-498f-8ffc-bec641c097d7-kube-api-access-zh6gs\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.608765 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.608614    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-lib-modules\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.608765 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.608649    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-podres\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.608765 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.608669    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-sys\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.608883 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.608769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-proc\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.608883 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.608780    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-lib-modules\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.608883 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.608825    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-sys\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.608883 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.608854    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2cfddf9c-f96e-498f-8ffc-bec641c097d7-proc\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.617816 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.617785    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh6gs\" (UniqueName: \"kubernetes.io/projected/2cfddf9c-f96e-498f-8ffc-bec641c097d7-kube-api-access-zh6gs\") pod \"perf-node-gather-daemonset-czjqs\" (UID: \"2cfddf9c-f96e-498f-8ffc-bec641c097d7\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.760783 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.760680    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:28.899541 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:28.899515    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"]
Jul 20 16:37:28.901542 ip-10-0-134-220 kubenswrapper[2564]: W0720 16:37:28.901512    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2cfddf9c_f96e_498f_8ffc_bec641c097d7.slice/crio-851df0a1046339eb2b5de37caf3b648b0c8f8f59c640eade7b7b51c58df74eea WatchSource:0}: Error finding container 851df0a1046339eb2b5de37caf3b648b0c8f8f59c640eade7b7b51c58df74eea: Status 404 returned error can't find the container with id 851df0a1046339eb2b5de37caf3b648b0c8f8f59c640eade7b7b51c58df74eea
Jul 20 16:37:29.212971 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:29.212936    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs" event={"ID":"2cfddf9c-f96e-498f-8ffc-bec641c097d7","Type":"ContainerStarted","Data":"7ad5369f812e26eaf1d66fea934e4510f12aed6746a01f1bbc005438e8d5736c"}
Jul 20 16:37:29.213157 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:29.212992    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs" event={"ID":"2cfddf9c-f96e-498f-8ffc-bec641c097d7","Type":"ContainerStarted","Data":"851df0a1046339eb2b5de37caf3b648b0c8f8f59c640eade7b7b51c58df74eea"}
Jul 20 16:37:29.213157 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:29.213024    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:29.232929 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:29.232875    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs" podStartSLOduration=1.232857804 podStartE2EDuration="1.232857804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:37:29.230657853 +0000 UTC m=+2813.885921133" watchObservedRunningTime="2026-07-20 16:37:29.232857804 +0000 UTC m=+2813.888121079"
Jul 20 16:37:31.478592 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:31.478551    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-btzkj_1ecd35be-c0a9-4936-8b02-97bf6b446e03/dns/0.log"
Jul 20 16:37:31.502993 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:31.502959    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-btzkj_1ecd35be-c0a9-4936-8b02-97bf6b446e03/kube-rbac-proxy/0.log"
Jul 20 16:37:31.597534 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:31.597505    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4lbp5_d0ab2581-3283-415f-b373-d3eedae55bfa/dns-node-resolver/0.log"
Jul 20 16:37:32.301184 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:32.301152    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-69d78676d4-mds9q_ecc6170f-e8c0-4ac4-ac53-729123ea019b/registry/0.log"
Jul 20 16:37:32.406311 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:32.406257    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dvh7c_a518917c-09da-43c3-b61e-f17a704fa9a2/node-ca/0.log"
Jul 20 16:37:33.799036 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:33.799006    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hpbz4_1a0c8fb0-d84c-432c-96f3-55f0e5312188/discovery/0.log"
Jul 20 16:37:33.883015 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:33.882990    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bc6c54cc6-rb9cf_8f0545fb-6ba1-45c6-8fac-d6717a671111/kube-auth-proxy/0.log"
Jul 20 16:37:34.027731 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:34.027678    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lfsj2_805b55dd-0b56-4831-8165-29ef025d779a/payload-processing/0.log"
Jul 20 16:37:34.813095 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:34.813067    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fpklh_a085a9d9-de82-432f-aa9e-c82a87d42243/serve-healthcheck-canary/0.log"
Jul 20 16:37:35.227478 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:35.227402    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-czjqs"
Jul 20 16:37:35.915381 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:35.915354    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j5hj8_70a70572-b756-4cda-9434-35357c5a2024/kube-rbac-proxy/0.log"
Jul 20 16:37:35.941664 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:35.941639    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j5hj8_70a70572-b756-4cda-9434-35357c5a2024/exporter/0.log"
Jul 20 16:37:35.976861 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:35.976835    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j5hj8_70a70572-b756-4cda-9434-35357c5a2024/extractor/0.log"
Jul 20 16:37:38.548747 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:38.548715    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9xmww_589ce640-c965-4058-87c3-e2e3eb813352/manager/0.log"
Jul 20 16:37:38.889587 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:38.889556    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pr94l_523b88b3-dd29-495c-b568-d04767ffe45a/manager/0.log"
Jul 20 16:37:39.073654 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:39.073621    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76d8696695-zq55p_ba52a7b9-3b00-4304-9891-a43d97f47cf4/manager/0.log"
Jul 20 16:37:39.332681 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:39.332644    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b76897bc6-f8cng_623fe211-2e84-4558-bbda-856abfa7a705/manager/0.log"
Jul 20 16:37:39.344054 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:39.344017    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hpbz4_1a0c8fb0-d84c-432c-96f3-55f0e5312188/discovery/0.log"
Jul 20 16:37:39.352382 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:39.352354    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b76897bc6-f8cng_623fe211-2e84-4558-bbda-856abfa7a705/copy-manifests/0.log"
Jul 20 16:37:39.373663 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:39.373630    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bc6c54cc6-rb9cf_8f0545fb-6ba1-45c6-8fac-d6717a671111/kube-auth-proxy/0.log"
Jul 20 16:37:39.475404 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:39.475373    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lfsj2_805b55dd-0b56-4831-8165-29ef025d779a/payload-processing/0.log"
Jul 20 16:37:41.009867 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:41.009825    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hpbz4_1a0c8fb0-d84c-432c-96f3-55f0e5312188/discovery/0.log"
Jul 20 16:37:41.047678 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:41.047645    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bc6c54cc6-rb9cf_8f0545fb-6ba1-45c6-8fac-d6717a671111/kube-auth-proxy/0.log"
Jul 20 16:37:41.163015 ip-10-0-134-220 kubenswrapper[2564]: I0720 16:37:41.162972    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lfsj2_805b55dd-0b56-4831-8165-29ef025d779a/payload-processing/0.log"